DSG Modeling and Optimization Guide¶

This notebook provides an overview of design space modeling and optimization capabilities using the DSG:

Create a DSG and define choices and constraints
Formulate an optimization problem from a DSG
Implement an evaluation function

Refer to the theory for more background information. For more elaborate examples, refer to the example notebooks in the left-side menu.

For reference, we display all the possible nodes and edges in the DSG here:

In [1]:

Copied!

from adsg_core.render import DSGRenderer
DSGRenderer.render_legend()
from adsg_core.render import DSGRenderer
DSGRenderer.render_legend()

Modeling Selection Choices¶

A selection choice node represents an architectural choice where one of the mutually-exclusive option nodes is selected. When resolved, the singular incoming generic node is connected to the selected option node by a derivation edge. The nodes not selected and their derived nodes, excluding confirmed nodes (see below), are removed from the graph. A selection choice can be ordinal (meaning there is inherent ordering between the options) or not.

A derivation edge is a directed edge is like a "requires" relationship, as it ensures that the target node is selected if the source node is selected.

One or more nodes are designated as start nodes: these nodes and their derived nodes are present in all architectures and therefore are designated permanent nodes. Non-permanent nodes are known as conditional.

An incompatibility edge is an undirected edge that asserts that if either of the two nodes is confirmed, the other node and its derived nodes are not. Incompatibility edges can lead to an infeasible DSG if both nodes are permanent.

We now show a simple example of defining a DSG containing several edges, selection choices, and an incompatibility constraint.

In [2]:

Copied!





from adsg_core import BasicDSG, NamedNode

# Create the DSG
dsg = BasicDSG()

# Create 10 nodes to work with
n = [NamedNode(f'N{i}') for i in range(10)]

# Add some edges
dsg.add_edges([
    (n[0], n[1]),  # N0 derives N1
    (n[3], n[4]),  # N3 derives N4
    (n[2], n[5]), (n[3], n[5]),  # N5 is derived by N2 and/or N3
])

# Add two selection choices:
# Connect N1 to N2, N3 or N4 (ordinal choice)
# Connect N5 to N6 or N7 (categorical choice)
dsg.add_selection_choice('C1', n[1], [n[2], n[3], n[4]], is_ordinal=True)
dsg.add_selection_choice('C2', n[5], [n[6], n[7]])

# Add incompatibility between N3 and N7
dsg.add_incompatibility_constraint([n[3], n[7]])

# Set the start node N0
# This function returns a modified DSG!
dsg = dsg.set_start_nodes({n[0]})

# Show the DSG (also works outside a notebook)
dsg.render(print_svg=False)
dsg.render_legend(['NODE', 'NODE_START', 'EDGE_DERIVE', 'CHOICE_SEL', 'EDGE_INCOMP'])
from adsg_core import BasicDSG, NamedNode

# Create the DSG
dsg = BasicDSG()

# Create 10 nodes to work with
n = [NamedNode(f'N{i}') for i in range(10)]

# Add some edges
dsg.add_edges([
    (n[0], n[1]),  # N0 derives N1
    (n[3], n[4]),  # N3 derives N4
    (n[2], n[5]), (n[3], n[5]),  # N5 is derived by N2 and/or N3
])

# Add two selection choices:
# Connect N1 to N2, N3 or N4 (ordinal choice)
# Connect N5 to N6 or N7 (categorical choice)
dsg.add_selection_choice('C1', n[1], [n[2], n[3], n[4]], is_ordinal=True)
dsg.add_selection_choice('C2', n[5], [n[6], n[7]])

# Add incompatibility between N3 and N7
dsg.add_incompatibility_constraint([n[3], n[7]])

# Set the start node N0
# This function returns a modified DSG!
dsg = dsg.set_start_nodes({n[0]})

# Show the DSG (also works outside a notebook)
dsg.render(print_svg=False)
dsg.render_legend(['NODE', 'NODE_START', 'EDGE_DERIVE', 'CHOICE_SEL', 'EDGE_INCOMP'])

Like this, we have defined a DSG with starting node N0 and two choices, C1 and C2. C2 is only activated if for C1 either N2 or N3 are chosen (because they derive N5). N4 is included in a graph instance both if N3 or N4 are chosen for C1.

Let's render all graph instances now. Observe that there are no more choice nodes, and that the selected nodes correspond to the output above. Also, there is no instance containing both and N3 and N7 are per the incompatibility constraint.

In [3]:

Copied!

dsg.render_all()  # Note: only use if there are not too many instances!
dsg.render_all()  # Note: only use if there are not too many instances!

Rendering 4 instances

Selection Choice Constraints¶

Choice constraints allow constraining option availability for choices based on other choices. For continuous design variables only linking is possible: here the same value relative to the respective design variable bounds is applied. For discrete design variables and selection and connection choices, four types of choice constraints are available: linked, permutations, unordered combinations, and unordered non-replacing combinations. These constraints enforce the following logic:

Linked: all choices are assigned the same option index, e.g. AA, BB, CC.
Permutations: all choices have a different option index, e.g. AB, AC, BA, BC, CA, CB.
Unordered combinations: choices have an equal or higher index than preceding choices, e.g. AA, AB, AC, BB, BC, CC.
Unordered non-replacing combinations: choices have a higher index than preceding choices, e.g. AB, AC, BC.

Here we show how to apply this for selection constraints.

In [4]:

Copied!





from adsg_core import ChoiceConstraintType

# Create a new DSG
dsg = BasicDSG()

# Create 2 independent choices with 3 options each
nn = [NamedNode(f'Opt{i}') for i in range(3)]
c1 = dsg.add_selection_choice('C1', n[0], nn)
c2 = dsg.add_selection_choice('C2', n[1], nn)

# Start nodes have to be defined before constraining the choices
dsg = dsg.set_start_nodes({n[0], n[1]})

# Constrain it so that only unordered non-replacing combinations are possible
# This function also returns a (potentially) modified DSG!
# "Unordered non-replacing" constraints are shown by purple-dashed edges annotated with ">"
dsg = dsg.constrain_choices(ChoiceConstraintType.UNORDERED_NOREPL, [c1, c2])
dsg.render()

dsg.render_all()
from adsg_core import ChoiceConstraintType

# Create a new DSG
dsg = BasicDSG()

# Create 2 independent choices with 3 options each
nn = [NamedNode(f'Opt{i}') for i in range(3)]
c1 = dsg.add_selection_choice('C1', n[0], nn)
c2 = dsg.add_selection_choice('C2', n[1], nn)

# Start nodes have to be defined before constraining the choices
dsg = dsg.set_start_nodes({n[0], n[1]})

# Constrain it so that only unordered non-replacing combinations are possible
# This function also returns a (potentially) modified DSG!
# "Unordered non-replacing" constraints are shown by purple-dashed edges annotated with ">"
dsg = dsg.constrain_choices(ChoiceConstraintType.UNORDERED_NOREPL, [c1, c2])
dsg.render()

dsg.render_all()

Rendering 3 instances

You can observe that all combinations of the option nodes are included, however no permutations, and no cases where the same options are selected.

Modeling Connection Choices¶

Connection choices offer a generic way to model source to target connection problems, where source and target nodes are represented using connection nodes. Connection nodes behave the same as generic nodes with respect to derivation edges and selection choice, however additionally specify a connector constraint: a specification of how many outgoing (source) or incoming (target) connections the connector node can accept, and whether parallel connections to/from the same target/source are allowed. The connector constraint can be specified as:

A list of numbers (e.g. 1, 2 or 3 connections: 1,2,3),
A lower and an upper bound (e.g. between 0 and 3, inclusive: 0..3),
Or only a lower bound (e.g. 1 or more: 1..*).

In the DSG, a connection choice is defined by adding connection edges from one or more source nodes to a connection choice node, and from the connection choice node to one or more target nodes. To model the case where the order of connections is not important, connection grouping nodes can be used: the connector constraint of this node depends on aggregated connector constraints of incoming connection nodes (connected by derivation edges). It is also possible to define combinations of source and target nodes that may not be connected using exclusion edges.

We now show a simple example of defining a DSG containing a connection choice.

In [5]:

Copied!





from adsg_core import BasicDSG, NamedNode, ConnectorNode

# Create the DSG and a start node
dsg = BasicDSG()
start = NamedNode('Start')

# Define connector nodes:
# Each needs at least one connection, with no upper limit
# Parallel (repeated) connections between the same src/tgt are not allowed
connectors = [ConnectorNode(f'CN{i}', deg_min=1, repeated_allowed=False)
              for i in range(4)]

# Add a connection choice connecting 2 sources to 2 targets
c = dsg.add_connection_choice(
    'C', src_nodes=connectors[:2], tgt_nodes=connectors[2:])

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in connectors])
dsg = dsg.set_start_nodes({start})
dsg.render()
dsg.render_legend(['NODE_CONNECTOR', 'EDGE_DERIVE', 'EDGE_CONNECT', 'CHOICE_CONN'])
from adsg_core import BasicDSG, NamedNode, ConnectorNode

# Create the DSG and a start node
dsg = BasicDSG()
start = NamedNode('Start')

# Define connector nodes:
# Each needs at least one connection, with no upper limit
# Parallel (repeated) connections between the same src/tgt are not allowed
connectors = [ConnectorNode(f'CN{i}', deg_min=1, repeated_allowed=False)
              for i in range(4)]

# Add a connection choice connecting 2 sources to 2 targets
c = dsg.add_connection_choice(
    'C', src_nodes=connectors[:2], tgt_nodes=connectors[2:])

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in connectors])
dsg = dsg.set_start_nodes({start})
dsg.render()
dsg.render_legend(['NODE_CONNECTOR', 'EDGE_DERIVE', 'EDGE_CONNECT', 'CHOICE_CONN'])

Let's see how many valid connection sets are possible

In [6]:

Copied!

n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There are {n_valid_conn} valid connection sets')

# Render two example instances
dsg.render_all([0, 1])
n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There are {n_valid_conn} valid connection sets')

# Render two example instances
dsg.render_all([0, 1])

There are 7 valid connection sets

Rendering 2 of 7 instances

Connection Grouping¶

Now let's model the case where for the target nodes, the connection order is irrelevant. That means that if CN1 connects to CN2 and CN3, we treat this as the same architecture as if it connects to CN3 and CN2.

In [7]:

Copied!





from adsg_core import ConnectorDegreeGroupingNode

# Create a new DSG
dsg = BasicDSG()

# Create the connection choice
# Here we add a connection grouping node to the targets
c = dsg.add_connection_choice(
    'C', src_nodes=connectors[:2],
    tgt_nodes=[
        # The list of target nodes can still also contain individual nodes
        # To specify a grouping node, we need to associate it with
        # the underlying connectors
        (ConnectorDegreeGroupingNode(), connectors[2:]),
    ],
)

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in connectors])
dsg = dsg.set_start_nodes({start})
dsg.render()
from adsg_core import ConnectorDegreeGroupingNode

# Create a new DSG
dsg = BasicDSG()

# Create the connection choice
# Here we add a connection grouping node to the targets
c = dsg.add_connection_choice(
    'C', src_nodes=connectors[:2],
    tgt_nodes=[
        # The list of target nodes can still also contain individual nodes
        # To specify a grouping node, we need to associate it with
        # the underlying connectors
        (ConnectorDegreeGroupingNode(), connectors[2:]),
    ],
)

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in connectors])
dsg = dsg.set_start_nodes({start})
dsg.render()

You can observe that the grouping node connection constraint covers both the individual connection constraints of C2 and C3. Now there is only one valid connection set left:

In [8]:

Copied!

n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There is {n_valid_conn} valid connection set')
dsg.render_all()
n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There is {n_valid_conn} valid connection set')
dsg.render_all()

There is 1 valid connection set

Rendering 1 instances

Connector Constraints¶

Let's see some more ways to define connector constraints.

In [9]:

Copied!





# Create a new DSG
dsg = BasicDSG()

# Define connector nodes with various constraints
src_conn = [
    ConnectorNode('S1', deg_list=[1],
                  repeated_allowed=True),  # 1 connection, parallel allowed
    ConnectorNode('S2', deg_list=[0, 1]),  # 1 optional connection
    ConnectorNode('S3', deg_min=0, deg_max=2),  # Max 2 connections
    ConnectorNode('S4', deg_min=1, deg_max=3),  # Between 1 and 3 connections
]
tgt_conn = [
    ConnectorNode('T1', deg_spec='*'),  # Any nr of connections
    ConnectorNode('T1', deg_spec='+'),  # At least 1 connection
    ConnectorNode('T1', deg_spec='?'),  # 1 optional connection
]

# Create the connection choice
c = dsg.add_connection_choice('C', src_nodes=src_conn, tgt_nodes=tgt_conn)

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in src_conn])
dsg.add_edges([(start, cn) for cn in tgt_conn])
dsg = dsg.set_start_nodes({start})
dsg.render()

n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There are {n_valid_conn} valid connection sets')
# Create a new DSG
dsg = BasicDSG()

# Define connector nodes with various constraints
src_conn = [
    ConnectorNode('S1', deg_list=[1],
                  repeated_allowed=True),  # 1 connection, parallel allowed
    ConnectorNode('S2', deg_list=[0, 1]),  # 1 optional connection
    ConnectorNode('S3', deg_min=0, deg_max=2),  # Max 2 connections
    ConnectorNode('S4', deg_min=1, deg_max=3),  # Between 1 and 3 connections
]
tgt_conn = [
    ConnectorNode('T1', deg_spec='*'),  # Any nr of connections
    ConnectorNode('T1', deg_spec='+'),  # At least 1 connection
    ConnectorNode('T1', deg_spec='?'),  # 1 optional connection
]

# Create the connection choice
c = dsg.add_connection_choice('C', src_nodes=src_conn, tgt_nodes=tgt_conn)

# Ensure all connector nodes are selected
dsg.add_edges([(start, cn) for cn in src_conn])
dsg.add_edges([(start, cn) for cn in tgt_conn])
dsg = dsg.set_start_nodes({start})
dsg.render()

n_valid_conn = len(list(c.iter_conn_edges(dsg)))
print(f'There are {n_valid_conn} valid connection sets')

There are 260 valid connection sets

Combining Selection and Connection Choices¶

Selection choices determine which nodes are present in an architecture instance, and connector nodes are also subject to this selection. Selection choices therefore may influence which valid connection sets are available for a connection choice.

We demonstrate the interaction using the example on the theory page.

In [10]:

Copied!





# Create the DSG and the start nodes
dsg = BasicDSG()
n = [NamedNode(f'N{i}') for i in range(2)]

# Define the connector nodes
src_nodes = [
    ConnectorNode('S1', deg_list=[1, 2], repeated_allowed=True),
    ConnectorNode('S2', deg_list=[1, 2], repeated_allowed=True),
    ConnectorNode('S3', deg_min=0, repeated_allowed=True),
]
tgt_nodes = [
    ConnectorNode('T1', deg_list=[1]),
    ConnectorNode('T2', deg_list=[0, 2], repeated_allowed=True),
]

# Add the selection choice and derivation edges
dsg.add_selection_choice('C1', n[0], src_nodes[:2])
dsg.add_edges([
    (n[0], src_nodes[2]),  # N0 to S3
    (src_nodes[1], src_nodes[0]),  # S2 to S1
    (n[1], tgt_nodes[0]), (n[1], tgt_nodes[1]),  # N1 to T1 and T2
])

# Add the connection choice
dsg.add_connection_choice(
    'C2', src_nodes=[
        (ConnectorDegreeGroupingNode('Grp'), src_nodes[:2]),
        src_nodes[2],
    ], tgt_nodes=tgt_nodes)

dsg = dsg.set_start_nodes({n[0], n[1]})
dsg.render()
# Create the DSG and the start nodes
dsg = BasicDSG()
n = [NamedNode(f'N{i}') for i in range(2)]

# Define the connector nodes
src_nodes = [
    ConnectorNode('S1', deg_list=[1, 2], repeated_allowed=True),
    ConnectorNode('S2', deg_list=[1, 2], repeated_allowed=True),
    ConnectorNode('S3', deg_min=0, repeated_allowed=True),
]
tgt_nodes = [
    ConnectorNode('T1', deg_list=[1]),
    ConnectorNode('T2', deg_list=[0, 2], repeated_allowed=True),
]

# Add the selection choice and derivation edges
dsg.add_selection_choice('C1', n[0], src_nodes[:2])
dsg.add_edges([
    (n[0], src_nodes[2]),  # N0 to S3
    (src_nodes[1], src_nodes[0]),  # S2 to S1
    (n[1], tgt_nodes[0]), (n[1], tgt_nodes[1]),  # N1 to T1 and T2
])

# Add the connection choice
dsg.add_connection_choice(
    'C2', src_nodes=[
        (ConnectorDegreeGroupingNode('Grp'), src_nodes[:2]),
        src_nodes[2],
    ], tgt_nodes=tgt_nodes)

dsg = dsg.set_start_nodes({n[0], n[1]})
dsg.render()

In [11]:

Copied!

# There should be 8 valid architectures (see table on the theory page)

# We render one of them that has multiple edges from S3 to T2
dsg.render_all([1])
# There should be 8 valid architectures (see table on the theory page)

# We render one of them that has multiple edges from S3 to T2
dsg.render_all([1])

Rendering 1 of 8 instances

Additional Design Variables and Metrics¶

Next to selection and connection choices, it is also possible to define generic design variables, for example to model parameter selections. These are defined using design variable nodes that are subject to node selection just as generic nodes and can therefore exist conditionally. Two types of design variables can be defined:

Continuous design variables, defined by a lower and upper bound (inclusive),
Discrete design variables, defined by a list of option values (strings).
- Discrete design variables are optionally ordinal, meaning that there is an inherent ordering between the option values. If not, the variable is categorical.

The design problem definition is completed by additionally defining performance metrics using metric nodes. Metric nodes represent outputs of an evaluation function and can be used as objectives or constraints in the context of an optimization problem:

Objectives are minimization or maximization targets. Metric nodes can only be used as objectives if they are permanent, as otherwise it is not possible to compare the performance of all architectures.
Constraints represent inequality design constraints: values that should be above (greater than or equal) or below (lower than or equal) some threshold. Metrics used as constraints can be conditional: if the node is not part of some architecture, it means that the constraint does not apply and the constraint is assumed satisfied (the value is set equal to the threshold).

In [12]:

Copied!





from adsg_core import BasicDSG, NamedNode, DesignVariableNode, \
    MetricNode, MetricType, GraphProcessor

# Create the DSG and some nodes
dsg = BasicDSG()
n = [NamedNode(f'N{i}') for i in range(10)]

# Define a selection choice
dsg.add_selection_choice('C1', n[0], [n[1], n[2]])
dsg.add_edge(n[0], n[3])

# Define an additional design variable (continuous)
dsg.add_edge(n[3], DesignVariableNode('DV1', bounds=(0, 1)))

# Define a conditional additional design variables (discrete)
dsg.add_edge(n[1], DesignVariableNode('DV2', options=['Red', 'Green', 'Blue']))
dsg.add_edge(n[1], DesignVariableNode('DV3', options=['A', 'B', 'C'], is_ordinal=True))

# Define a generic metric node
dsg.add_edge(n[1], MetricNode('Metric'))

# Define an objective (should be permanent)
dsg.add_edge(n[3], MetricNode('Objective', direction=-1,
                              type_=MetricType.OBJECTIVE))

# Define a constraint (can be conditional)
dsg.add_edge(n[2], MetricNode('Constraint', direction=1, ref=2,
                              type_=MetricType.CONSTRAINT))

# Set start nodes and render
dsg = dsg.set_start_nodes({n[0]})
dsg.render()

# Check their roles in an optimization problem
processor = GraphProcessor(dsg)
nl = '\n'
print(f'Design variables:{nl}{nl.join("  "+str(dv) for dv in processor.des_vars)}')
print(f'Objectives:{nl}{nl.join("  "+str(obj) for obj in processor.objectives)}')
print(f'Constraints:{nl}{nl.join("  "+str(con) for con in processor.constraints)}')
print('')

dsg.render_legend(['NODE_START', 'EDGE_DERIVE', 'CHOICE_SEL', 'METRICS'])
from adsg_core import BasicDSG, NamedNode, DesignVariableNode, \
    MetricNode, MetricType, GraphProcessor

# Create the DSG and some nodes
dsg = BasicDSG()
n = [NamedNode(f'N{i}') for i in range(10)]

# Define a selection choice
dsg.add_selection_choice('C1', n[0], [n[1], n[2]])
dsg.add_edge(n[0], n[3])

# Define an additional design variable (continuous)
dsg.add_edge(n[3], DesignVariableNode('DV1', bounds=(0, 1)))

# Define a conditional additional design variables (discrete)
dsg.add_edge(n[1], DesignVariableNode('DV2', options=['Red', 'Green', 'Blue']))
dsg.add_edge(n[1], DesignVariableNode('DV3', options=['A', 'B', 'C'], is_ordinal=True))

# Define a generic metric node
dsg.add_edge(n[1], MetricNode('Metric'))

# Define an objective (should be permanent)
dsg.add_edge(n[3], MetricNode('Objective', direction=-1,
                              type_=MetricType.OBJECTIVE))

# Define a constraint (can be conditional)
dsg.add_edge(n[2], MetricNode('Constraint', direction=1, ref=2,
                              type_=MetricType.CONSTRAINT))

# Set start nodes and render
dsg = dsg.set_start_nodes({n[0]})
dsg.render()

# Check their roles in an optimization problem
processor = GraphProcessor(dsg)
nl = '\n'
print(f'Design variables:{nl}{nl.join("  "+str(dv) for dv in processor.des_vars)}')
print(f'Objectives:{nl}{nl.join("  "+str(obj) for obj in processor.objectives)}')
print(f'Constraints:{nl}{nl.join("  "+str(con) for con in processor.constraints)}')
print('')

dsg.render_legend(['NODE_START', 'EDGE_DERIVE', 'CHOICE_SEL', 'METRICS'])

Design variables:
  DV: C1 [2 opts]
  DV: DV1 [0.00..1.00]
  DV: DV2 [3 opts]
  DV: DV3 [3 opts, ord]
Objectives:
  OBJ: Objective [min]
Constraints:
  CON: Constraint [>= 2.00]

Defining Optimization Problems¶

An architecture optimization problem can be modeled with the DSG using:

Generic nodes, derivation edges, start nodes and selection choice nodes to define node existence hierarchies.
Incompatibility constraints to restrict simultaneous existence of nodes.
Connection (grouping) nodes with connector constraints, connection edges, exclusion edges and connection choice nodes to define connection problems (source to target connections).
Generic design variable nodes to define additional continuous or discrete design variables.
Choice constraints to restrict possible values within a group of choices or design variables.
Metric nodes to define output metrics, optionally used as objectives or constraints.

Encoding a DSG into an optimization problem is done using encoding algorithms, for more information refer to the theory. We now demonstrate how to encode a DSG as an optimization and use that to run an optimization. We use the Jenatton problem as a running example: Design variables:

x1, x2, x3: categorical design variables (0 or 1)
x4 to x9: continuous design variables (0 to 1)

Objective:

if x1 == 0:
  if x2 == 0:
    f = x4^2 + 0.1 + x8
  else:
    f = x5^2 + 0.1 + x8
else:
  if x3 == 0:
    f = x6^2 + 0.1 + x9
  else:
    f = x7^2 + 0.1 + x9

In [13]:

Copied!





from adsg_core import BasicDSG, NamedNode, DesignVariableNode, \
    MetricNode, MetricType

# Create DSG
dsg = BasicDSG()
start = NamedNode('Start')

# Add x1 choice
x1_0, x1_1 = NamedNode('x1=0'), NamedNode('x1=1')
dsg.add_selection_choice('x1', start, [x1_0, x1_1])

# Add x2 choice (if x1 = 0)
x2_0, x2_1 = NamedNode('x2=0'), NamedNode('x2=1')
dsg.add_selection_choice('x1', x1_0, [x2_0, x2_1])

# Add x3 choice (if x1 = 1)
x3_0, x3_1 = NamedNode('x3=0'), NamedNode('x3=1')
dsg.add_selection_choice('x1', x1_1, [x3_0, x3_1])

# Add design variable nodes and objective
dsg.add_edges([
    (x2_0, DesignVariableNode('x4', bounds=(0, 1))),
    (x2_1, DesignVariableNode('x5', bounds=(0, 1))),
    (x3_0, DesignVariableNode('x6', bounds=(0, 1))),
    (x3_1, DesignVariableNode('x7', bounds=(0, 1))),
    (x1_0, DesignVariableNode('x8', bounds=(0, 1))),
    (x1_1, DesignVariableNode('x9', bounds=(0, 1))),
    (start, MetricNode('f', direction=-1, type_=MetricType.OBJECTIVE)),
])

# Set start node and render
dsg = dsg.set_start_nodes({start})
dsg.render()
from adsg_core import BasicDSG, NamedNode, DesignVariableNode, \
    MetricNode, MetricType

# Create DSG
dsg = BasicDSG()
start = NamedNode('Start')

# Add x1 choice
x1_0, x1_1 = NamedNode('x1=0'), NamedNode('x1=1')
dsg.add_selection_choice('x1', start, [x1_0, x1_1])

# Add x2 choice (if x1 = 0)
x2_0, x2_1 = NamedNode('x2=0'), NamedNode('x2=1')
dsg.add_selection_choice('x1', x1_0, [x2_0, x2_1])

# Add x3 choice (if x1 = 1)
x3_0, x3_1 = NamedNode('x3=0'), NamedNode('x3=1')
dsg.add_selection_choice('x1', x1_1, [x3_0, x3_1])

# Add design variable nodes and objective
dsg.add_edges([
    (x2_0, DesignVariableNode('x4', bounds=(0, 1))),
    (x2_1, DesignVariableNode('x5', bounds=(0, 1))),
    (x3_0, DesignVariableNode('x6', bounds=(0, 1))),
    (x3_1, DesignVariableNode('x7', bounds=(0, 1))),
    (x1_0, DesignVariableNode('x8', bounds=(0, 1))),
    (x1_1, DesignVariableNode('x9', bounds=(0, 1))),
    (start, MetricNode('f', direction=-1, type_=MetricType.OBJECTIVE)),
])

# Set start node and render
dsg = dsg.set_start_nodes({start})
dsg.render()

Encoding the Design Space¶

The DSG is encoded into an optimization problem using the GraphProcessor class.

In [14]:

Copied!





from adsg_core import GraphProcessor

gp = GraphProcessor(dsg)

print('Design variables:', gp.des_vars)
print('Objectives:', gp.objectives)
print('Constraints:', gp.constraints)

# Display some details about the encoders used for
# formulating the optimization problem
gp.get_statistics()
from adsg_core import GraphProcessor

gp = GraphProcessor(dsg)

print('Design variables:', gp.des_vars)
print('Objectives:', gp.objectives)
print('Constraints:', gp.constraints)

# Display some details about the encoders used for
# formulating the optimization problem
gp.get_statistics()

Design variables: [DV: x1 [2 opts], DV: x1_2 [2 opts], DV: x1_3 [2 opts], DV: x4 [0.00..1.00], DV: x5 [0.00..1.00], DV: x6 [0.00..1.00], DV: x7 [0.00..1.00], DV: x8 [0.00..1.00], DV: x9 [0.00..1.00]]
Objectives: [OBJ: f [min]]
Constraints: []

Out[14]:

	n_valid	n_declared	n_discrete	n_dim_cont	n_dim_cont_mean	n_exist	imp_ratio	imp_ratio_comb	imp_ratio_cont	inf_idx	dist_corr	encoder
type
option-decisions	4	8	3	0	0.0	1	2.0	2.0	1.0	1.0	0.0	complete
additional-dvs	4	0	0	6	2.0	4	3.0	1.0	3.0	1.0	0.0
total-design-space	4	8	3	6	2.0	1	6.0	2.0	3.0	1.0	0.0	complete
total-design-problem	4	8	3	6	2.0	1	6.0	2.0	3.0	1.0	0.0	complete

We can then use the GraphProcessor to:

Generate DSG instances from a design vector.
Generate all valid discrete design vectors.
Get some statistics about the design problem.

In [15]:

Copied!





# Generate random design vector and turn it into an DSG instance
random_x = gp.get_random_design_vector()
dsg_instance, x_corrected, is_active = gp.get_graph(random_x)

# The design vector might be corrected
print(f'x input: {random_x!r}')
print(f'x corrected: {x_corrected!r}')
print(f'activeness: {is_active}')

dsg_instance.render()
print('x values:', dsg_instance.des_var_values)
# Generate random design vector and turn it into an DSG instance
random_x = gp.get_random_design_vector()
dsg_instance, x_corrected, is_active = gp.get_graph(random_x)

# The design vector might be corrected
print(f'x input: {random_x!r}')
print(f'x corrected: {x_corrected!r}')
print(f'activeness: {is_active}')

dsg_instance.render()
print('x values:', dsg_instance.des_var_values)

x input: [1, 0, 1, 0.4949998942752789, 0.6822325329360271, 0.9104277529327189, 0.4551464905052941, 0.019395881794497583, 0.5988961436760696]
x corrected: [1, 0, 1, 0.5, 0.5, 0.5, 0.4551464905052941, 0.5, 0.5988961436760696]
activeness: [True, False, True, False, False, False, True, False, True]

x values: {DV[x7]: 0.4551464905052941, DV[x9]: 0.5988961436760696}

Observe that the inactive design variables (given by the activeness vector) are "imputed" to a canonical value. For continuous variable that is mid-bounds, which in this case is 0.5.

In [16]:

Copied!

# Generate all valid discrete design vectors
x_all, is_active_all = gp.get_all_discrete_x()
x_all  # Print all design vectors
# Generate all valid discrete design vectors
x_all, is_active_all = gp.get_all_discrete_x()
x_all  # Print all design vectors

Out[16]:

array([[0. , 0. , 0. , 0. , 0.5, 0.5, 0.5, 0. , 0.5],
       [0. , 1. , 0. , 0.5, 0. , 0.5, 0.5, 0. , 0.5],
       [1. , 0. , 0. , 0.5, 0.5, 0. , 0.5, 0.5, 0. ],
       [1. , 0. , 1. , 0.5, 0.5, 0.5, 0. , 0.5, 0. ]])

In [17]:

Copied!

is_active_all  # Print associated activeness information
is_active_all  # Print associated activeness information

Out[17]:

array([[ True,  True, False,  True, False, False, False,  True, False],
       [ True,  True, False, False,  True, False, False,  True, False],
       [ True, False,  True, False, False,  True, False, False,  True],
       [ True, False,  True, False, False, False,  True, False,  True]])

The GraphProcessor can also be used to fix certain design variables. This can be useful for ad-hoc narrowing of the design space.

In [18]:

Copied!





dv = gp.des_vars

# Fix the first design variable
gp.fix_des_var(dv[0], 0)

# Observe that there are now only two valid discrete design vectors left
print(f'Valid x:', gp.get_all_discrete_x()[0].shape[0], '(x0 is fixed)')

# Free the design variable again
gp.free_des_var(dv[0])
print(f'Valid x:', gp.get_all_discrete_x()[0].shape[0])
dv = gp.des_vars

# Fix the first design variable
gp.fix_des_var(dv[0], 0)

# Observe that there are now only two valid discrete design vectors left
print(f'Valid x:', gp.get_all_discrete_x()[0].shape[0], '(x0 is fixed)')

# Free the design variable again
gp.free_des_var(dv[0])
print(f'Valid x:', gp.get_all_discrete_x()[0].shape[0])

Valid x: 2 (x0 is fixed)
Valid x: 4

Defining the Evaluation Function¶

The last part needed because we can run the optimization problem is the performance evaluation function: the function that for a given DSG instance, returns its performance in terms of the defined output metrics. This allows the optimization algorithm to iteratively search the design space and find the optimum.

An evaluation function should fulfill the following requirements:

Performance metrics should be sensitive to all relevant choices.
Performance metrics should be available for all relevant architectures, and with similar accuracy/fidelity. Objectives should always be available; constraints can be optional.
The evaluation function should be executable without user interaction.

The evaluation function is implemented by defining a new evaluator class that inherits from ADSGEvaluator. There, the _evaluate function should be implemented. This function takes as input:

The DSG instance being evaluated.
A list of performance metrics for which output is requested.

The function should return a dictionary mapping performance metrics to float values. It is also possible to use NaN as a value, signifying that the associated metric could not be calculated.

In [19]:

Copied!





from typing import List, Dict
from adsg_core import DSGEvaluator, ADSGType

# Extend the DSGEvaluator class and implement _evaluate
class JenattonEvaluator(DSGEvaluator):

    def __init__(self):
        # Use the previously defined DSG...
        # When implementing your own evaluator, it probably makes
        # more sense to define the DSG in some class method
        super().__init__(dsg)

    def _evaluate(self, dsg_inst: ADSGType, metric_nodes: List[MetricNode])\
            -> Dict[MetricNode, float]:
        # In general the equation is a^2 + 0.1 + b
        # a is x4 to x7; b is x8 or x9

        # Loop over design variable nodes to find the values
        a = b = None
        for dv_node in dsg_inst.des_var_nodes:
            if dv_node.name in {'x8', 'x9'}:
                b = dsg_inst.des_var_value(dv_node)
            else:
                a = dsg_inst.des_var_value(dv_node)
        assert a is not None and b is not None

        # Calculate the objective
        f = a**2 + .1 + b

        # Return the metric
        assert len(metric_nodes) == 1
        return {metric_nodes[0]: f}
from typing import List, Dict
from adsg_core import DSGEvaluator, ADSGType

# Extend the DSGEvaluator class and implement _evaluate
class JenattonEvaluator(DSGEvaluator):

    def __init__(self):
        # Use the previously defined DSG...
        # When implementing your own evaluator, it probably makes
        # more sense to define the DSG in some class method
        super().__init__(dsg)

    def _evaluate(self, dsg_inst: ADSGType, metric_nodes: List[MetricNode])\
            -> Dict[MetricNode, float]:
        # In general the equation is a^2 + 0.1 + b
        # a is x4 to x7; b is x8 or x9

        # Loop over design variable nodes to find the values
        a = b = None
        for dv_node in dsg_inst.des_var_nodes:
            if dv_node.name in {'x8', 'x9'}:
                b = dsg_inst.des_var_value(dv_node)
            else:
                a = dsg_inst.des_var_value(dv_node)
        assert a is not None and b is not None

        # Calculate the objective
        f = a**2 + .1 + b

        # Return the metric
        assert len(metric_nodes) == 1
        return {metric_nodes[0]: f}

Now we can use the evaluator to evaluate the performance of a DSG instance.

In [20]:

Copied!





# Instantiate the evaluator
evaluator = JenattonEvaluator()

# Get a DSG instance (same API as the GraphProcessor)
x = [1, 0, 0, 0.5, 0.5, 0.29, 0.5, 0.5, 0.58]
dsg_instance, _, _ = evaluator.get_graph(x)
dsg_instance.render()
print('x values:', dsg_instance.des_var_values)

expected_obj = .29**2 + .1 + .58

# Evaluate the instance
obj, con = evaluator.evaluate(dsg_instance)
print(f'Objective: {obj[0]} (expected: {expected_obj})')
dsg_instance.render()
# Instantiate the evaluator
evaluator = JenattonEvaluator()

# Get a DSG instance (same API as the GraphProcessor)
x = [1, 0, 0, 0.5, 0.5, 0.29, 0.5, 0.5, 0.58]
dsg_instance, _, _ = evaluator.get_graph(x)
dsg_instance.render()
print('x values:', dsg_instance.des_var_values)

expected_obj = .29**2 + .1 + .58

# Evaluate the instance
obj, con = evaluator.evaluate(dsg_instance)
print(f'Objective: {obj[0]} (expected: {expected_obj})')
dsg_instance.render()

x values: {DV[x6]: 0.29, DV[x9]: 0.58}
Objective: 0.7641 (expected: 0.7641)

Running the Optimization Problem¶

Now that we have modeled the design space, defined the optimization, and implemented the evaluation function, we can run the optimization problem:

optimization loop

We run the optimization problem by coupling to SBArchOpt. SBArchOpt is an open-source library for running architecture optimization problem. A problem definition in the API of SBArchOpt is provided, so that all algorithms defined in SBArchOpt can be used.

Ensure SBArchOpt is installed, or run: pip install adsg-core[opt]

In [21]:

Copied!





# Get the SBArchOpt problem
problem = evaluator.get_problem()

# Now the SBArchOpt API is available
# We can for example print some problem statistic
problem.print_stats()
# Get the SBArchOpt problem
problem = evaluator.get_problem()

# Now the SBArchOpt API is available
# We can for example print some problem statistic
problem.print_stats()

problem: DSGArchOptProblem(<__main__.JenattonEvaluator object at 0x000001C532E97340>)
n_discr: 3
n_cont : 6
n_obj  : 1
n_con  : 0
MD     : True
MO     : False
HIER         : True
n_valid_discr: 4
imp_ratio    : 6.00 (discr.: 2.00; cont.: 3.00)
corr_ratio   : 3.00 (discr.: 1.00; cont.: 3.00; fraction of imp_ratio: 61.3%)
                     x0    x1    x2   max
inactive                  0.5   0.5      
opt 0               0.5   0.5   0.5      
opt 1               0.5   0.5   0.5      
diversity           0.0  0.25  0.25  0.25
active-diversity    0.0   0.0   0.0   0.0
x_type              cat   cat   cat      
is_cond           False  True  True      
                      n_valid  n_declared  n_discrete  n_dim_cont  n_dim_cont_mean  n_exist  imp_ratio  imp_ratio_comb  imp_ratio_cont  inf_idx  dist_corr   encoder
type                                                                                                                                                                
option-decisions            4           8           3           0              0.0        1        2.0             2.0             1.0      1.0        0.0  complete
additional-dvs              4           0           0           6              2.0        4        3.0             1.0             3.0      1.0        0.0          
total-design-space          4           8           3           6              2.0        1        6.0             2.0             3.0      1.0        0.0  complete
total-design-problem        4           8           3           6              2.0        1        6.0             2.0             3.0      1.0        0.0  complete

We now run the optimization using the NSGA-II algorithm. For more information refer to the SBArchOpt documentation. SBArchOpt is based on pymoo, so it can also be helpful to consult its documentation.

In [22]:

Copied!





from pymoo.optimize import minimize
from sb_arch_opt.algo.pymoo_interface import get_nsga2

# Get the optimization algorithm
algorithm = get_nsga2(pop_size=100)

# Run the optimization
result = minimize(problem, algorithm, termination=('n_gen', 20), verbose=True)

# Print results
opt = result.opt
print('Best f:', opt.get('F')[0])
print('Best x:', list(opt.get('X')[0]))
from pymoo.optimize import minimize
from sb_arch_opt.algo.pymoo_interface import get_nsga2

# Get the optimization algorithm
algorithm = get_nsga2(pop_size=100)

# Run the optimization
result = minimize(problem, algorithm, termination=('n_gen', 20), verbose=True)

# Print results
opt = result.opt
print('Best f:', opt.get('F')[0])
print('Best x:', list(opt.get('X')[0]))

==========================================================================================================================
n_gen  |  n_eval  | n_nds  |      eps      |   indicator   |     hv_est    |   not_failed  |    feasible   |    optimal   
==========================================================================================================================
     1 |      100 |      1 |             - |             - | -0.000000E+00 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     2 |      200 |      1 |  0.000000E+00 |             f | -0.000000E+00 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     3 |      300 |      1 |  0.0519015664 |         ideal |  0.0519015664 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     4 |      400 |      1 |  0.0004848177 |             f |  0.0523863842 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     5 |      500 |      1 |  0.0004848177 |             f |  0.0523863842 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     6 |      600 |      1 |  0.0523227197 |         ideal |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     7 |      700 |      1 |  0.000000E+00 |             f |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     8 |      800 |      1 |  0.000000E+00 |             f |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
     9 |      900 |      1 |  0.000000E+00 |             f |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    10 |     1000 |      1 |  0.000000E+00 |             f |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    11 |     1100 |      1 |  0.000000E+00 |             f |  0.1042242861 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    12 |     1200 |      1 |  0.0137629025 |         ideal |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    13 |     1300 |      1 |  0.000000E+00 |             f |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    14 |     1400 |      1 |  0.000000E+00 |             f |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    15 |     1500 |      1 |  0.000000E+00 |             f |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    16 |     1600 |      1 |  0.000000E+00 |             f |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    17 |     1700 |      1 |  0.000000E+00 |             f |  0.1179871886 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    18 |     1800 |      1 |  0.0003227879 |             f |  0.1183099766 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    19 |     1900 |      1 |  0.0005396081 |             f |  0.1185267967 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
    20 |     2000 |      1 |  0.0019903857 |             f |  0.1199775744 |  100 (100.0%) |  100 (100.0%) |      1 (1.0%)
Best f: [0.1002921]
Best x: [1.0, 0.0, 0.0, 0.5, 0.5, 0.01493039571501716, 0.5, 0.5, 6.918309340119347e-05]