Diagram-as-Code: Creating Dynamic and Interactive Documentation for Visual Content
#python#programming#aws#documentation
Romina Mendez

Romina Mendez @r0mymendez

About: Data Architect & Data Engineer - University Professor

Joined:
Oct 5, 2021

Diagram-as-Code: Creating Dynamic and Interactive Documentation for Visual Content

Publish Date: Nov 19 '24
529 32

In this article, I will guide you step by step to create dynamic and interactive visual documentation using Diagram-as-Code tools. Instead of static images, we will generate diagrams programmatically, ensuring they are always up-to-date and easy to maintain.

Image description

🎨 Diagram as code

Diagram as Code is an approach that allows you to create diagrams through code instead of traditional graphic tools. Instead of manually building diagrams, you can write code in a text file to define the structure, components, and connections of your diagrams.

This code is then translated into graphical images, making it easier to integrate and document in software projects, where it is especially useful for creating and updating architectural and flow diagrams programmatically.

What is Diagrams?

Diagrams is a 🐍Python library that implements the Diagram as Code approach, enabling you to create architectural infrastructure diagrams and other types of diagrams through code. With Diagrams, you can easily define cloud infrastructure components (such as AWS, Azure, and GCP), network elements, software services, and more, all with just a few lines of code.

🎉 Benefits of Diagram-as-Code

  • 📝 Representation of Diagrams as Code: Create and update diagrams directly from code, ensuring maintainability in agile projects.
  • 📑 Automated Documentation: Generate visuals from code, keeping diagrams aligned with the current architecture.
  • 🔄 Change Control: Track diagram modifications over time.
  • 🔍 Enhanced Clarity: Improve understanding of complex systems with clear, shared visuals.
  • ✏️ Customizable: Represent cloud infrastructures, workflows, or data pipelines with flexible and tailored visuals.

Tutorial

🐍 Library Installation

I was currently using version '0.23.4' for this tutorial.

!pip install diagrams=='0.23.4'
Enter fullscreen mode Exit fullscreen mode

🎨 Diagrams: Nodes

The library allows you to create architectural diagrams programmatically, using nodes to represent different infrastructure components and services.

Node Types

Nodes in Diagrams represent components from different cloud service providers as well as other architectural elements. Here are the main categories of available nodes:

  • ☁️ Cloud Providers: AWS (Amazon Web Services), Azure, GCP, IBM Cloud, Alibaba Cloud, Oracle Cloud, DigitalOcean, among others.
  • 🏢 On-Premise: Represents the infrastructure physically located on the company's premises.
  • 🚢 Kubernetes (K8S): Container orchestration system to automate the deployment, scaling, and management of containerized applications (represented by a ship's wheel, symbolizing control and navigation).
  • 🖥️ OpenStack: Open-source software platform for creating and managing public and private clouds.
  • 🔧 Generic: Generic nodes that can represent any component not specifically covered by provider-specific nodes (crossed tools, representing different tools in one category).
  • ☁️ SaaS (Software as a Service): Represents applications delivered as a service over the internet, such as Snowflake, chat services (Slack, Teams, Telegram, among others), security (e.g., Okta), or social networks (crossed out phone and cloud for the SaaS concept).
  • 🔧 Custom: Allows users to customize their diagrams using PNG icons stored in a specific folder. This is useful for representing infrastructure components not covered by the default nodes (crossed-out custom tools).

💻 Programming Languages

The Diagrams library allows you to use different nodes to represent various programming languages. These nodes are helpful for indicating in your diagrams if any part of your architecture utilizes scripts or components developed in a specific programming language.

Below, we will showcase all the available languages in the library. If any language is missing, you can add custom nodes by uploading the corresponding logo into a specific folder.

# Create the diagram object
with diagrams.Diagram("Programming Languages", show=False, filename="languages"):
    # Get all the languages available in this library
    languages = [item for item in dir(diagrams.programming.language) if item[0] != '_']

    # Divide the representation in two lines
    mid_index = len(languages) // 2
    first_line = languages[:mid_index]
    second_line = languages[mid_index:]

    # Add nodes in the first row
    prev_node = None

    for language in first_line:
        current_node = eval(f"diagrams.programming.language.{language}(language)")
        if prev_node is not None:
            prev_node >> current_node
        prev_node = current_node

    # Add nodes in the second row
    prev_node = None

    for language in second_line:
        current_node = eval(f"diagrams.programming.language.{language}(language)")
        if prev_node is not None:
            prev_node >> current_node
        prev_node = current_node

Image("languages.png")
Enter fullscreen mode Exit fullscreen mode

Image description

☁️ AWS (Amazon Web Services)

We can use Amazon nodes, which are organized into several categories, such as:

  • Analytics and Business: aws.analytics, aws.business
  • Compute and Storage: aws.compute, aws.storage, aws.cost
  • Database and DevTools: aws.database, aws.devtools
  • Integration and Management: aws.integration, aws.management
  • Machine Learning and Mobile: aws.ml, aws.mobile
  • Networking and Security: aws.network, aws.security
  • Others: aws.blockchain, aws.enduser, aws.engagement, aws.game, aws.general, aws.iot, aws.media, aws.migration, aws.quantum, aws.robotics, aws.satellite

Next, we will represent one of these categories to visualize the available nodes within aws.database.

from diagrams import Diagram
from IPython.display import Image
import diagrams.aws.database as aws_database


database_components = []
for item in dir(aws_database):
    if item[0] != '_':
        if not any(comp.startswith(item) or item.startswith(comp) for comp in database_components):
            database_components.append(item)


with Diagram("AWS Database", show=False, filename="aws_database"):
    mid_index = len(database_components) // 2
    first_line = database_components[:mid_index]
    second_line = database_components[mid_index:]


    prev_node = None
    for item_database in first_line:
        current_node = eval(f"aws_database.{item_database}(item_database)")
        if prev_node is not None:
            prev_node >> current_node
        prev_node = current_node


    prev_node = None
    for item_database in second_line:
        current_node = eval(f"aws_database.{item_database}(item_database)")
        if prev_node is not None:
            prev_node >> current_node
        prev_node = current_node

Image("aws_database.png")
Enter fullscreen mode Exit fullscreen mode

Image description

☁️ Use Case

Now, let's create a simple blueprint that corresponds to importing a dataset and training a machine learning model on AWS.

from diagrams import Diagram, Cluster
from diagrams.aws.storage import S3
from diagrams.aws.analytics import Glue, Athena
import diagrams.aws.ml as ml
from diagrams.aws.integration import StepFunctions
from diagrams.aws.compute import Lambda
from diagrams.aws.network import APIGateway
from IPython.display import Image

with Diagram("AWS Data Processing Pipeline", show=False):

    lambda_raw = Lambda('Get Raw Data')
    # Buckets de S3
    with Cluster("Data Lake"):
        s3_rawData = S3("raw_data")
        s3_stage = S3("staging_data")
        s3_data_capture = S3("data_capture")


    athena = Athena("Athena")
    s3_rawData >> athena
    s3_stage >> athena
    s3_data_capture >> athena

    #  Step Functions Pipeline
    with Cluster("Data Processing Pipeline"):
        step_functions = StepFunctions("Pipeline")

        # Glue Jobs in Step Functions
        with Cluster("Glue Jobs"):
            data_quality = Glue("job_data_quality")
            transform = Glue("job_data_transform")
            dataset_preparation = Glue("job_dataset_model")

        # Define Step Functions Flows
        step_functions >> data_quality >> transform >> dataset_preparation
        s3_rawData >> data_quality

    # SageMaker for model training and deployment
    with Cluster("SageMaker Model Deployment"):
        train_model = ml.SagemakerTrainingJob("job_train_model")
        eval_model = ml.SagemakerGroundTruth("job_evaluate_model")
        endpoint = ml.SagemakerModel("model_enpoint")

    # API Gateway and Lambda for the endpoint
    api_gateway = APIGateway("API_gateway")
    lambda_fn = Lambda("invoke_endpoint")

    # Connection
    lambda_raw >> s3_rawData
    s3_stage >> train_model >> eval_model >> endpoint
    endpoint >> lambda_fn >> api_gateway
    endpoint >> s3_data_capture
    dataset_preparation >> train_model


Image("aws_data_processing_pipeline.png")
Enter fullscreen mode Exit fullscreen mode

Image description

Repository

Below are the link to all the code, if you find it useful, you can leave a star ⭐️ and follow me to receive notifications of new articles. This will help me grow in the tech community and create more content.

GitHub logo r0mymendez / diagram-as-code

A tutorial on how to create a documentation project using the 'Doc as diagram' methodology

Buy Me A Coffee

🎨 Diagram-as-Code: Creating Dynamic and Interactive Documentation for Visual Content

Diagram as Code is an approach that allows you to create diagrams through code instead of traditional graphic tools. Instead of manually building diagrams, you can write code in a text file to define the structure, components, and connections of your diagrams.

Image description

This code is then translated into graphical images, making it easier to integrate and document in software projects, where it is especially useful for creating and updating architectural and flow diagrams programmatically.

What is Diagrams?

Diagrams is a 🐍Python library that implements the Diagram as Code approach, enabling you to create architectural infrastructure diagrams and other types of diagrams through code. With Diagrams, you can easily define cloud infrastructure components (such as AWS, Azure, and GCP), network elements, software services, and more, all with just a few lines of code.

🎉 Benefits of Diagram-as-Code

  • 📝…


View on GitHub


If you want to see how to implement a documentation site using this pipeline you can read the article I published in the following link

📚 References

  1. Diagrams: https://diagrams.mingrammer.com/

Comments 32 total

  • Srinivasulu Paranduru
    Srinivasulu ParanduruNov 20, 2024

    Nice one @r0mymendez

  • Vibhor Mahajan
    Vibhor MahajanNov 21, 2024

    I’ve used Plantuml with AWS icons in the past github.com/awslabs/aws-icons-for-p...

    Plantuml is very versatile and has great tooling support with the IDEs providing preview capabilities.

    Should I be considering this python library if I’m a Plantuml user?

  • Fidelis Ikoroje
    Fidelis IkorojeNov 21, 2024

    Good one. I will try this

  • Charles J. Fowler
    Charles J. FowlerNov 21, 2024

    Mermaid

  • Edward Snodgrass
    Edward SnodgrassNov 21, 2024

    Thanks for sharing. I'm going to give this a try.

  • Prajwal KS
    Prajwal KSNov 21, 2024

    Nice project

  • Rebeccca Peltz
    Rebeccca PeltzNov 21, 2024

    Thanks for sharing!

  • picklerick63
    picklerick63Nov 21, 2024

    Really interesting, thank you. I couldn't work out how bitwise operator >> was playing a part until I looked up and found that there's an operator overload at play here and this operator represents a connection aka edge between the components!

  • Gokul krishnaa
    Gokul krishnaaNov 22, 2024

    Thanks for sharing the article! Very useful.

  • Emmanuel Valverde Ramos
    Emmanuel Valverde RamosNov 22, 2024

    Do you know c4model.com/

    • Romina Mendez
      Romina MendezNov 22, 2024

      No, I didn't know about it, but thanks for sharing it

  • dami
    damiNov 22, 2024

    This is great

  • ANIRUDDHA ADAK
    ANIRUDDHA ADAKNov 23, 2024

    Absolutely incredible! 😻.

  • Oshan Methsara
    Oshan MethsaraNov 23, 2024

    Best Programming Codes to Sell
    Get the best programming codes — 5000+ codes to buy or download for free!

    Best Programming Solutions
    _Explore thousands of programming codes in JavaScript, Python, and PHP. Whether you're building your next web app, data analysis model, or CMS plugin, we have it all.

    5000+ codes available for you to buy or download for free! Start building today!_

    👉 bestprolanguagecodes.blogspot.com 👈

  • Bikash Daga
    Bikash DagaNov 24, 2024

    Nice read!

  • Biswajit Patra
    Biswajit PatraNov 25, 2024

    Thanks for sharing about a grate product.
    I was looking for something like this.

  • Dhanush
    DhanushNov 26, 2024

    This approach is especially useful for creating and updating architectural and flow diagrams programmatically, particularly in cloud infrastructure environments like AWS.

  • Amit Wani
    Amit WaniNov 26, 2024

    Next step is to make it AI generated. Based on prompt

    • Romina Mendez
      Romina MendezNov 26, 2024

      Yes! It is a good idea 💡

      • Amit Wani
        Amit WaniNov 26, 2024

        I am building a chart generator similar to this based on data and prompt. MakeChart.app

  • Ceno
    CenoDec 2, 2024

    thanks for sharing

  • Nestor Gutiérrez
    Nestor GutiérrezDec 6, 2024

    I like this because you can keep the diagram in the same repo, can always keep it up to date and anyone that clones the repo can have the diagram, I want to use it to diagram some workflows in my project.

    • Romina Mendez
      Romina MendezDec 6, 2024

      Yes! It's a great idea because you can also track the evolution of the architecture over time. Plus, if you ever need to reuse a previous version of the diagram for another project, it's easy to do with Git's version control.

  • Tu Do
    Tu DoDec 18, 2024

    Did this support export as SVG file?

  • Iasmim Oliveira
    Iasmim OliveiraJan 22, 2025

    really awesome! i was looking for a library like that

  • Madhurima Rawat
    Madhurima RawatJan 23, 2025

    Thanks for this article! I had no idea a library like this existed—what a fantastic resource! 😊 The repository is amazing too—already starred it! ⭐ Keep up the incredible work; you're doing great! 🚀

Add comment