Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions doc/manual/chapters/create.adoc
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
There are several ways to create a new simulation scenario in NSE2, depending on your preferences and requirements.
This section covers the three main methods: using the `netedit` tool, converting CSV/JSON files, and manually creating a scenario file.
Depending on whether you prefer a graphical interface, command-line tools, or manual editing, you can choose the method that best suits your workflow.
The recommended way to create an NSE2 scenario is to start with a scenario in
the https://github.com/esa/ccsds-dtn-reference-scenarios[CCSDS DTN Reference
Scenarios] format. The repository provides the scenario data as CSV files;
these can be converted directly into the NSE2 contact plan and Docker Compose
files. The generated files keep node names and links synchronized, avoiding the
need to understand the specifics of the <<contactplan-format,CCP format>> and
<<compose-format,Compose format>>.

The CSV scenario can be used unchanged, adapted before conversion, or used as
the basis for a more substantial modification. Alternatively, an already
converted scenario from NSE2's `scenarios/` directory can be copied and
adapted. Creating both files manually should generally be avoided, but remains
useful for small examples or special-purpose topologies. `netedit` is another
alternative for graphical topology editing, though its GraphML conversion to NSE2
files is currently not yet supported.
155 changes: 94 additions & 61 deletions doc/manual/chapters/create/convert.adoc
Original file line number Diff line number Diff line change
@@ -1,89 +1,122 @@
== Generate Files from CSV
The following commands implement the CCSDS CSV workflow described above. Use
the same contact CSV to generate both a contact plan and a Docker Compose
topology.

The helper scripts in `tools/helpers` can generate a contact plan and a
Docker Compose file from the same contact CSV. The CSV contains one link
per row with the columns `source`, `destination`, `start`, `end`,
`bandwidth`, `delay`, and an optional label. The scripts skip the header
row.
The generators are available as the `csv_to_ccp` and `csv_to_compose` commands
(or directly under `tools/helpers/` in a source checkout).

=== Generate a Contact Plan
==== Contact CSV

Use `csv_to_ccp.py` to create a `.ccp` file:
The input CSV has a header followed by one row per directed link:

[cols="1,1,1,1,1,1,1",options="header"]
|===
| Source | Destination | Start | End | Bandwidth | Delay | Label

| Node name | Node name | Seconds | Seconds | Bits per second | Milliseconds | Optional
|===

The header row is skipped. The label column is optional. Loss and jitter are
not CSV columns and are generated as zero. A row with `start=0` and `end=-1`
becomes a fixed link. Matching rows in opposite directions are combined into a
symmetric link where their timing and link properties match.

Use the `Label` column to distinguish multiple links between the same pair of
nodes when they have different properties. Generated Docker network and
interface names are based on the node names and, where needed, the link label.
Keep node names short; if a generated name exceeds Linux's 15-character
interface-name limit, it is replaced by a 12-character MD5 prefix. A long
common prefix can sometimes be avoided with `--strip-prefix PREFIX`; link
labels ending in `high` or `low` are shortened to `hi` or `lo` when names are
constructed.

==== Generate a Contact Plan

[source,bash]
----
csv_to_ccp.py contacts.csv --output contacts.ccp
csv_to_ccp contacts.csv --output contacts.ccp
----

Options:
The available options are:

* `--output`, `-o`: output file; `-` (the default) writes to standard output.
* `--output`, `-o`: output file; `-` writes to standard output.
* `--strip-prefix PREFIX`: remove `PREFIX` from source and destination names.
* `--speedup FACTOR`: divide contact timestamps by `FACTOR`, useful for a
shorter test run. The default is `1`.
* `--bw-scale FACTOR` (also `--bandwidth-scale`): multiply bandwidths by
`FACTOR`. The default is `1`.

Rows with a start time of `0` and an end time of `-1` become fixed links.
Matching reverse-direction rows are combined into a symmetric link using
`=`. When a node pair has multiple labels, the label is used to generate a
dedicated destination interface such as `dev:n1_n2_payload`.
* `--speedup FACTOR`: divide contact times by `FACTOR` to shorten a scenario.
* `--bw-scale FACTOR` or `--bandwidth-scale FACTOR`: scale bandwidth values.

=== Generate a Compose File
When multiple labelled links connect the same pair of nodes, the label is used
to select a destination interface such as `dev:n1_n2_payload`.

Use `csv_to_compose.py` with the same CSV:
==== Generate a Compose Topology

[source,bash]
----
csv_to_compose.py contacts.csv --output compose.yml
csv_to_compose contacts.csv --output compose.yml
----

Options:
The optional node mapping JSON supplies stable metadata for each CSV node. Its
entries contain `node_label`, `node_name`, and `node_id`:

* `--output`, `-o`: output file; `-` writes to standard output.
* `--strip-prefix PREFIX`: remove `PREFIX` from node names.
* `--nodes FILE` (also `--mapping`): JSON file containing node labels, names,
and IDs.
* `--name NAME`, `-n`: Compose project name.
* `--image IMAGE`, `-i`: container image; the default is `alpine`.
* `--entrypoint COMMAND`, `-e`: container entrypoint. The default installs
`iproute2` and `bash`, then keeps the container running.
* `--build PATH`, `-b`: use a build directory instead of an image.
* `--base-subnet PREFIX`: first two subnet octets; the default is `172.33`.
* `--export-graphml FILE`, `-g`: additionally export the topology as GraphML.
* `--node-volumes DIRECTORY`: create per-node data directories and mount them
into the containers.
* `--mount-compose`: mount the generated Compose file read-only in each
container.

The generated Compose topology uses one Docker bridge network for each link.
Every such network connects exactly two nodes; this keeps each link isolated
and gives both endpoints a predictable interface. Network names are based on
the sorted node names, and, when several differently labelled links connect
the same pair, also include the link label. Direction suffixes `_ul` and
`_dl` are removed and `high`/`low` are shortened to `hi`/`lo`.

Linux limits network interface names to 15 characters including the
terminating null byte. Therefore names with 14 or more characters are
replaced by the first 12 characters of their MD5 hash, with a warning printed
by the script. Use `--strip-prefix` when node labels share a long common
prefix, for example:
[source,json]
----
[
{
"node_label": "n1",
"node_name": "spacecraft-1",
"node_id": "ipn:1.0"
}
]
----

If a mapping is not supplied, metadata is generated from the CSV labels. Use
the mapping when the CCSDS scenario provides stable names and IDs.

The available `csv_to_compose` options are:

* `--output`, `-o`: Compose output file; `-` writes to standard output.
* `--strip-prefix PREFIX`: remove `PREFIX` from node labels.
* `--nodes FILE` or `--mapping FILE`: read node metadata from JSON.
* `--name NAME`, `-n`: set the Compose project name.
* `--image IMAGE`, `-i`: select the node image; the default is `alpine`.
* `--entrypoint COMMAND`, `-e`: override the default Alpine entrypoint.
* `--base-subnet PREFIX`: choose the first two subnet octets; the default is
`172.33`.
* `--node-volumes DIRECTORY`: create and mount a data directory per node.
* `--build PATH`, `-b`: build nodes from the specified path instead of an image.
* `--mount-compose`: mount the generated Compose file read-only in each node.
* `--export-graphml FILE`, `-g`: additionally write the generated topology as
GraphML. This is an output option; the script does not read GraphML as input.

The generator creates one Docker bridge network for each link, with predictable
interfaces for the endpoints.

==== Generate Both Files

Generate the two NSE2 inputs from the same CCSDS contact CSV:

[source,bash]
----
csv_to_compose.py --strip-prefix eo --nodes nodes.json \
--output compose.yml actual_contacts.csv
csv_to_ccp --strip-prefix eo actual_contacts.csv --output contacts.ccp
csv_to_compose --strip-prefix eo --nodes nodes.json \
--name earth-orbit actual_contacts.csv --output compose.yml
----

Removing the prefix can keep the generated network and interface names short
and human-readable instead of forcing the hash fallback.
Both generators record the generation date and command in their output. Review
the generated files before starting the scenario, then use the normal NSE2
workflow from <<running>>.

Both scripts write the generation date and complete command to the generated
file. A typical scenario therefore uses the same CSV to generate both files:
==== Shortening Realistic Contact Plans

As an alternative to `csv_to_ccp` 's time and bandwidth scaling, `random_contacts`
keeps the topology and link properties but replaces long, realistic contact
windows with short random ones for a compact test run:

[source,bash]
----
csv_to_ccp.py --strip-prefix eo --output contacts.ccp actual_contacts.csv
csv_to_compose.py --strip-prefix eo --nodes nodes.json \
--name earth-orbit --output compose.yml actual_contacts.csv
random_contacts contacts.ccp contacts-short.ccp \
--length 300 --min-contact 5 --max-contact 20 --seed 42
----

The `--length` option sets the plan duration; `--min-contact`, `--max-contact`,
and `--seed` control the generated windows. Fixed links and all link properties
are preserved.
11 changes: 7 additions & 4 deletions doc/manual/chapters/create/manually.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
Creating a new simulation scenario from scratch is a straight forward process, but it requires a good understanding of the Docker Compose file format and the specific requirements for network simulation. The following sections will guide you through the steps to manually create a scenario using Docker Compose.
Manual creation is mainly useful for small examples or special-purpose
topologies. The following sections describe the files that can make up an NSE2
scenario and how to create them using Docker Compose.

For a concrete example, you can refer to the simple scenario in the `scenarios/simple` directory of the NSE2 repository, which contains a basic setup with three nodes connected in a line.
It is described in detail in <<running>>, but you can also use it as a starting point for your own scenarios.
For a concrete example, refer to the simple scenario in the
`scenarios/simple` directory. It contains three nodes connected in a line and
is described in detail in <<running>>.

==== Create a Docker Compose File

Expand All @@ -25,4 +28,4 @@ If no actions are needed, the actions file can be empty or not used at all.

The visualization file is a JSON file that defines a scenario description, the nodes with their visualization properties, and where live link information can be found. It is used to visualize the network topology and the active links between the nodes.
The visualization file is typically named `viz.json` and is placed in the same directory as the Compose file. The details about the visualization file format can be found in <<viz-format>>.
If no visualization is needed, the visualization file can be empty or not used at all.
If no visualization is needed, the visualization file can be empty or not used at all.
9 changes: 3 additions & 6 deletions doc/manual/chapters/create/netedit.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,11 @@ TIP: In the link properties dialog, you can also set the link to be a dynamic li

NOTE: Some advanced NSE2 features such as multiple parallel links between the same pair of nodes, or using specific network interfaces for the links, are not supported by `netedit` yet. You can still use the `compose.yml` file to define these features manually.

Via the _File_ menu, you can save the current topology to a GraphML file, which can then be converted to a Docker Compose file.
Via the _File_ menu, you can save the current topology to a GraphML file.

==== Converting GraphML to Docker Compose

**TODO**

==== Running the Scenario

Once you have a proper Docker Compose file, you can run the scenario using the `nse2_topo` command. This command will start the Docker containers defined in the Compose file and set up the network topology as specified.

Of course, you can also add a contact plan, actions file, and visualization file as described in <<manually>> to enhance the simulation experience.
NSE2 currently has no helper scripts for converting GraphML to a Docker Compose
Comment thread
axodentally marked this conversation as resolved.
topology or a contact plan.
17 changes: 12 additions & 5 deletions doc/manual/chapters/formats/compose.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,19 @@ You set the container `image` or alternatively the `build` option to specify the

IMPORTANT: The container images used for the nodes must have `iproute2` installed so that `tc qdisc` can be used to configure the network emulation for the link effects. Also, for very large delays, e.g., several minutes like to mars, a version of `iproute2` from https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=9a6b231ea1b09e450688c5814a4c89a57cdbee77[mid 2024] or later is required.

There are a few optional things to configure for each node:
For scenarios managed by `nse2_contacts`, every node must define the
`environment` variable `NODE_ID` with a unique number. The contact player uses
this value when resolving node references from the contact plan and applying
its contacts. Applications can also use it for DTN auto-configuration.

- Sharing the compose and contact plan files `read-only` with the node container (`./compose.yml:/compose.yml:ro`, `./contacts.ccp:/contacts.ccp:ro`), so that the node can parse the topology if needed, for routing purposes.
- Set the `environment` variables `NODE_ID` to a unique number, e.g., to use for IPN auto configuration in DTN networks. This is useful for the node to identify itself in the simulation.
The Compose and contact plan files can optionally be shared with a node as
read-only mounts (`./compose.yml:/compose.yml:ro`,
`./contacts.ccp:/contacts.ccp:ro`). This is useful when an application inside
the node needs to inspect the topology or contact plan, for example to set up
its own routing. The mounts are not required by the contact player itself.

Of course, you can also add custom entry points or any other environment variables that are needed for the node to run, such as configuration files or other parameters specific to the service.
Custom entrypoints and additional environment variables can be added when the
node service needs application-specific configuration.

TIP: Sometimes it is useful to wrap your `entrypoint` commands in a `bash -c` command to ensure that the commands are executed in a shell environment. This is especially useful if you want to use shell features like variable expansion or command substitution. An example of this would be: `entrypoint: 'sh -c "apk add iproute2 bash && tail -f /dev/null"'`

Expand All @@ -45,4 +52,4 @@ Additionally, it can be helpful to set the `ipam` option to specify the IP addre

Finally, if you expect multiple links between the same pair of nodes, you can use specific network interfaces for the link (`com.docker.network.container_iface_prefix`). This prefix is used for the actual network interface name in the node container. Contact plans can then refer to these interfaces by their name, e.g., `dev:eth0` or `dev:n1_n2_0` for the second interface, and so on, instead of using the node name as second node.

WARNING: Network names in docker are global. Thus, running two scenarios with the same network names in parallel will lead to conflicts!
WARNING: Network names in docker are global. Thus, running two scenarios with the same network names in parallel will lead to conflicts!
3 changes: 1 addition & 2 deletions doc/manual/chapters/formats/viz.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Optionally, it can also put a background image behind the nodes to provide a bet
.Example visualization file of an earth observation scenario.
image::images/viz_eo.png[width=70%, align="center", alt="Example visualization file with nodes and links."]

The visualization frontend is started by running the `nse2_viz` command, which reads the `viz.json` file and starts a web server to serve the visualization.
The visualization frontend is started by running the `nse2_netviz` command, which reads the `viz.json` file and starts a web server to serve the visualization.

The `viz.json` file used for <<viz-eo-example>> is as follows:

Expand All @@ -32,4 +32,3 @@ The `links` key is used to specify the file that contains the current state of l
The file just contains a list of links, each pair on a separate line and a "-" or "." indicating whether it is a fixed or dynamic link, e.g., `n1 - n2` or `n1 . n2`.

IMPORTANT: To get a proper visualization of the active links, `nse2_contacts` must be started with the `-m` option to periodically generate a mapping file with all active links.

Loading
Loading