feat: add environment metadata file `sysand_env/env.toml` by victor-linroth-sensmetry · Pull Request #175 · sensmetry/sysand

victor-linroth-sensmetry · 2026-02-11T11:10:23Z

This PR adds a env.toml containing metadata for the projects installed in the local environment
(inluding editable projects not present inside the sysand_env folder).

Each project has the following fields:

publisher: String (optional).
Publisher of the project. Intended for display purposes.
name: String (optional).
Name of the project. Intended for display purposes.
version: String (required).
Version of the project.
path: String (required).
Path to the root directory of the project.
If the project is not editable this should be relative
to the env directory and otherwise it should be relative
to the workspace root.
identifiers: Array of strings (required for non-editable projects)
List of identifiers (IRIs) used for the project.
The first identifier is to. be considered the canonical
identifier, and if the project is not editable this
is the IRI it is installed as. The rest are considered
as aliases. Can only be empty for editable projects.
usages: Array of strings.
Usages of the project. Intended for tools needing to
track the interdependence of project in the environment.
editable: bool.
Indicator of wether the project is fully installed in
the environment or located elsewhere.
files: Array of strings (only for editable projects).
In case of an editable project these are the files
belonging to the project. Intended for tools that
are not able to natively parse and understand the
projects .meta.json file. Paths should be relative
to the path of the project.
workspace: Indicator of wether the project is part of a workspace.

The current implementation doesn't change how projects.txt works and is intended to offer a transition step between env structures. The env.toml isn't directly managed by
LocalDirectoryEnvironment since such a thing will likely require a change in the
WriteEnvironment trait. In particular it would have to offer the ability to give alias identifiers in addition to the main IRI used for installation (since we want to communicate the dependency structure between projects we need all aliases as well, at least if we want to incrementally add or remove from the environment).

Also since the current implementation of workspaces only affects the build command, the workspace field isn't really effective at the moment, but should work as expected once the workspace feature catches up.

Example:

# This file is automatically generated by Sysand and is not intended to be edited manually.

version = "0.1"

[[project]]
name = "SYSMOD"
identifiers = [
    "urn:kpar:sysmod",
]
version = "5.0.0-alpha.2"
path = "a0aacee34dd4cd5e2d07ab43d5e30772ec86dbf3c8fafb033bad338dd7a0f02e/5.0.0-alpha.2.kpar"
usages = [
    "urn:kpar:analysis-library",
    "urn:kpar:cause-and-effect-library",
    "urn:kpar:data-type-library",
    "urn:kpar:function-library",
    "urn:kpar:geometry-library",
    "urn:kpar:metadata-library",
    "urn:kpar:quantities-and-units-library",
    "urn:kpar:requirement-derivation-library",
    "urn:kpar:semantic-library",
    "urn:kpar:systems-library",
]

[[project]]
name = "Testing"
version = "0.0.1"
path = "."
usages = [
    "urn:kpar:sysmod",
]
editable = true
files = [
    "test.sysml",
    "test.kerml",
]

andrius-puksta-sensmetry · 2026-02-11T15:02:12Z

Information provided (if available) includes

Publisher

Name

Directory/Files

Usages

What about IRIs for non-PURL usages? Since names will likely not be unique (e.g. 10 projects named "Requirements"), how will they be disambiguated to the user?

victor-linroth-sensmetry · 2026-02-11T20:30:31Z

What about IRIs for non-PURL usages? Since names will likely not be unique (e.g. 10 projects named "Requirements"), how will they be disambiguated to the user?

Well, our solution for name collisions is to use namespaces(/publishers). If you don't use them, then you don't get the solution.

daumantas-kavolis-sensmetry · 2026-02-12T07:10:52Z

This is communicated through a "resolved manifest" current.toml

current.toml is very non-descriptive or sounds quite irrelevant to an environment. env.toml or projects.toml would be far clearer.

In particular this means paths are absolute

I highly suggest using paths relative to manifest/sysand_env for paths inside sysand_env and project folders as majority of the paths will have an identical prefix that can trivially be removed.

What about IRIs for non-PURL usages? Since names will likely not be unique (e.g. 10 projects named "Requirements"), how will they be disambiguated to the user?

Why would anyone care about collisions in what is effectively a display name? If a user has multiple projects named the same, they have bigger issues than some ambiguous name.

daumantas-kavolis-sensmetry · 2026-02-12T07:45:33Z

Maybe also add workspace = [0, 1, ...] top-level property that controls workspace/project roots for analysis?

core/src/env/local_directory/manifest.rs

…t of the current project. Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

andrius-puksta-sensmetry · 2026-03-25T11:42:56Z

core/src/env/local_directory/metadata.rs

+    #[serde(deserialize_with = "deserialize_unix_path")]
+    pub path: Utf8UnixPathBuf,
+    /// List of identifiers (IRIs) used for the project.
+    /// The first identifier is to. be considered the canonical


Suggested change

/// The first identifier is to. be considered the canonical

/// The first identifier is considered the canonical

andrius-puksta-sensmetry · 2026-03-25T11:43:43Z

core/src/env/local_directory/metadata.rs

+    /// to the `path` of the project.
+    #[serde(deserialize_with = "deserialize_optional_unix_paths", default)]
+    pub files: Option<Vec<Utf8UnixPathBuf>>,
+    /// Indicator of wether the project is part of a workspace.


Suggested change

/// Indicator of wether the project is part of a workspace.

/// Indicator of whether the project is part of current workspace.

andrius-puksta-sensmetry · 2026-03-25T11:48:34Z

files: Array of strings (only for editable projects).
In case of an editable project these are the files
belonging to the project. Intended for tools that
are not able to natively parse and understand the
projects .meta.json file. Paths should be relative
to the path of the project.

Since editable projects are expected to change frequently, how will
new/removed files be handled?

workspace: Indicator of wether the project is part of a workspace.

Which workspace? Workspace that owns this env? Or another unrelated one?

andrius-puksta-sensmetry · 2026-03-25T11:53:17Z

core/src/project/utils.rs

+                    .as_os_str()
+                    .to_str()
+                    // This conversion should always be possible since everything is UTF8 encoded
+                    .expect("component of `Utf8Path` no convertible to `str`"),


Suggested change

.expect("component of `Utf8Path` no convertible to `str`"),

.expect("component of `Utf8Path` not convertible to `str`"),

Nevermind, just use component.as_str() directly.

Yes, looking at it the str conversion is probably leftover code from before camino. Will fix.

victor-linroth-sensmetry · 2026-03-25T12:13:38Z

Since editable projects are expected to change frequently, how will new/removed files be handled?

Hmm, good question. Could maybe be updated whenever the env is updated. Or simply simply drop files and say tools will need to parse the .meta.json if they want exact files. I've been leaning towards that before.

Which workspace? Workspace that owns this env? Or another unrelated one?

Yeah, I think the idea at least is to have common lockfile, env and build directory in a workspace. This is to indicate of that is the case and which projects are workspace memebers.

andrius-puksta-sensmetry · 2026-03-25T12:23:54Z

Since editable projects are expected to change frequently, how will new/removed files be handled?

Hmm, good question. Could maybe be updated whenever the env is updated. Or simply simply drop files and say tools will need to parse the .meta.json if they want exact files. I've been leaning towards that before.

Agree on dropping files. It's not env responsibility to precisely keep track of projects outside of it.

Which workspace? Workspace that owns this env? Or another unrelated one?

Yeah, I think the idea at least is to have common lockfile, env and build directory in a workspace. This is to indicate of that is the case and which projects are workspace memebers.

Ok, so current workspace. Yeah, having common env/lockfile/build dir is a sensible thing to do.

andrius-puksta-sensmetry · 2026-03-25T12:29:35Z

core/src/lock.rs

+    // Simple stopgap solution for now
+    pub fn get_package_url<'a>(&self) -> Option<PackageUrl<'a>> {
+        self.identifiers
+            .first()
+            .and_then(|id| PackageUrl::from_str(id.as_str()).ok())
+    }


Shouldn't this use publisher and name (if both are present)?

Especially with my upcoming changes to usage specification, not sure if identifiers will contain the PURL (haven't decided yet).

Leftover from before publisher was added. Should probably remove.

andrius-puksta-sensmetry · 2026-03-25T12:49:19Z

sysand/src/commands/clone.rs

+    // TODO: Consider under which circumstances (if any)
+    //       the workspace should carry over.


Workspace should not carry over by default, as that's unlikely to be the goal of the user when cloning a project. It would be sufficient to maybe expose command to add current project to workspace (out of scope of this PR), but not sure if it's worth it, workspace file can be easily edited manually.

sysand/src/lib.rs

andrius-puksta-sensmetry · 2026-03-25T13:23:32Z

core/src/env/local_directory/metadata.rs

+                let root_path = if let Some(current_workspace) = &ctx.current_workspace {
+                    wrapfs::canonicalize(current_workspace.root_path())?
+                } else if let Some(current_project) = &ctx.current_project {
+                    wrapfs::canonicalize(current_project.root_path())?
+                } else {
+                    wrapfs::canonicalize(&ctx.current_directory)?
+                };


These should already be canonical and if not, should be made so elsewhere, as we need them to be canonical in most places. Regarding Windows, probably a good idea to use dunce crate to avoid UNC where possible.

Yeah, this was due to Windows and UNC nonsense. canonicalize and current_dir gives different formats for some reason.

From what I found, CWD is generally never UNC on Windows.

core/src/env/local_directory/metadata.rs

andrius-puksta-sensmetry · 2026-03-25T13:53:19Z

The env.toml isn't directly managed by
LocalDirectoryEnvironment since such a thing will likely require a change in the
WriteEnvironment trait. In particular it would have to offer the ability to give alias identifiers in addition to the main IRI used for installation (since we want to communicate the dependency structure between projects we need all aliases as well, at least if we want to incrementally add or remove from the environment).

Add a TODO to this effect somewhere. Not sure what to do about this longer term, as env.toml also includes editable projects, which are not installed into the environment per se, and also the current project(s)... WriteEnvironment then would have to be somehow coupled with current workspace/projects, not sure how to go about that.

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

victor-linroth-sensmetry · 2026-03-30T07:15:07Z

Add a TODO to this effect somewhere. Not sure what to do about this longer term, as env.toml also includes editable projects, which are not installed into the environment per se, and also the current project(s)... WriteEnvironment then would have to be somehow coupled with current workspace/projects, not sure how to go about that.

Put a TODO in command_sync as that's the main point of generating env.toml.

andrius-puksta-sensmetry · 2026-03-30T08:30:48Z

core/src/env/local_directory/metadata.rs

+#[derive(Debug, Deserialize)]
+pub struct EnvMetadata {
+    pub version: String,
+    #[serde(rename = "project", skip_serializing_if = "Vec::is_empty", default)]


No need for these here, as the serialization is implemented manually. Same applies to Lock (maybe there are others?).

andrius-puksta-sensmetry · 2026-03-30T08:31:50Z

core/src/env/local_directory/metadata.rs

+    /// identifier, and if the project is not `editable` this
+    /// is the IRI it is installed as. The rest are considered
+    /// as aliases. Can only be empty for `editable` projects.
+    #[serde(skip_serializing_if = "Vec::is_empty", default)]


No need for this, see above.

(same applies to other fields here)

andrius-puksta-sensmetry · 2026-03-30T08:32:34Z

core/src/env/local_directory/metadata.rs

+    pub fn to_toml(&self) -> Table {
+        let mut table = Table::new();
+        if let Some(publisher) = &self.publisher {
+            table.insert("publisher", value(publisher));
+        }
+        if let Some(name) = &self.name {
+            table.insert("name", value(name));
+        }
+        table.insert("version", value(&self.version));
+        table.insert("path", value(self.path.as_str()));
+        if !self.identifiers.is_empty() {
+            table.insert(
+                "identifiers",
+                value(multiline_array(self.identifiers.iter())),
+            );
+        }
+        if !self.usages.is_empty() {
+            table.insert("usages", value(multiline_array(self.usages.iter())));
+        }
+        if self.editable {
+            table.insert("editable", value(true));
+        }
+
+        table
+    }


workspace is ignored

andrius-puksta-sensmetry · 2026-03-30T08:37:52Z

core/src/env/local_directory/metadata.rs

+    /// Adds identifiers from other project.
+    /// Should only be done if the underlying projects are the same.
+    /// In particular they must have the same version.
+    pub fn merge(&mut self, other: &EnvProject) {


Suggested change

pub fn merge(&mut self, other: &EnvProject) {

pub fn merge_identifiers(&mut self, other: &EnvProject) {

andrius-puksta-sensmetry · 2026-03-30T08:43:27Z

core/src/project/utils.rs

+        dunce::canonicalize(path.as_ref())
+            .map(|path| {
+                Utf8PathBuf::from_path_buf(path)
+                    .expect("expected Dunce not to introduce non UTF8 characters")


This may not be the case with symlinks/relative path inputs. Do a manual match and return ErrorKind::InvalidData if conversion fails, like camino does.

andrius-puksta-sensmetry · 2026-03-30T08:51:49Z

sysand/src/commands/env.rs

 pub fn command_env<P: AsRef<Utf8Path>>(path: P) -> Result<LocalDirectoryEnvironment> {
    Ok(do_env_local_dir(path)?)
 }


This doesn't seem to create env.toml, which is later relied upon to exist in add_single_env_project. Am I missing something?

victor-linroth-sensmetry self-assigned this Feb 11, 2026

victor-linroth-sensmetry force-pushed the feat/resolved-manifest branch from f5eb786 to 2ff1dba Compare February 11, 2026 11:52

daumantas-kavolis-sensmetry reviewed Feb 12, 2026

View reviewed changes

core/src/env/local_directory/manifest.rs Outdated Show resolved Hide resolved

consideRatio changed the title ~~Add a resolved manifest to sysand_env~~ Add a resolved manifest sysand_env/current.toml Mar 13, 2026

andrius-puksta-sensmetry changed the title ~~Add a resolved manifest sysand_env/current.toml~~ dd a resolved manifest sysand_env/current.toml Mar 16, 2026

andrius-puksta-sensmetry changed the title ~~dd a resolved manifest sysand_env/current.toml~~ feat: add a resolved manifest sysand_env/current.toml Mar 16, 2026

victor-linroth-sensmetry force-pushed the feat/resolved-manifest branch from 2ff1dba to dea655c Compare March 20, 2026 07:53

victor-linroth-sensmetry changed the title ~~feat: add a resolved manifest sysand_env/current.toml~~ feat: add environment metadata file sysand_env/env.toml Mar 20, 2026

victor-linroth-sensmetry force-pushed the feat/resolved-manifest branch 8 times, most recently from 5f996f0 to ec6f714 Compare March 24, 2026 20:28

victor-linroth-sensmetry added 10 commits March 24, 2026 21:29

Adds a resolved manifest to sysand_env keeping track of what is par…

592b170

…t of the current project. Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Add optional publisher to manifest.

7f3b5ce

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Split up local_directory.

ac2b9fb

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Deserialize empty arrays.

7f5a863

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Go from current.toml -> env.toml and include more fields.

84189d3

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Update metadata format.

d197f76

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Make sure metadata is updated along with env.

f531fef

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Fix for tests on Ubuntu (ARM).

c459895

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Fix paths issues on Windows.

6faf3fc

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Use Unix paths.

baadc25

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Add workspace field.

51e123f

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

victor-linroth-sensmetry force-pushed the feat/resolved-manifest branch from ec6f714 to 51e123f Compare March 24, 2026 20:34

victor-linroth-sensmetry marked this pull request as ready for review March 25, 2026 07:43

andrius-puksta-sensmetry reviewed Mar 25, 2026

View reviewed changes

sysand/src/lib.rs Outdated Show resolved Hide resolved

andrius-puksta-sensmetry reviewed Mar 25, 2026

View reviewed changes

core/src/env/local_directory/metadata.rs Show resolved Hide resolved

andrius-puksta-sensmetry reviewed Mar 25, 2026

View reviewed changes

core/src/env/local_directory/metadata.rs Show resolved Hide resolved

andrius-puksta-sensmetry reviewed Mar 25, 2026

View reviewed changes

core/src/env/local_directory/metadata.rs Show resolved Hide resolved

andrius-puksta-sensmetry reviewed Mar 25, 2026

View reviewed changes

core/src/env/local_directory/metadata.rs Show resolved Hide resolved

victor-linroth-sensmetry added 2 commits March 27, 2026 10:56

Address feedback.

77e8763

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

Remove files field.

72a80ca

Signed-off-by: victor.linroth.sensmetry <victor.linroth@sensmetry.com>

andrius-puksta-sensmetry reviewed Mar 30, 2026

View reviewed changes

	/// The first identifier is to. be considered the canonical
	/// The first identifier is considered the canonical

	/// Indicator of wether the project is part of a workspace.
	/// Indicator of whether the project is part of current workspace.

	.expect("component of `Utf8Path` no convertible to `str`"),
	.expect("component of `Utf8Path` not convertible to `str`"),

		// TODO: Consider under which circumstances (if any)
		// the workspace should carry over.

	pub fn merge(&mut self, other: &EnvProject) {
	pub fn merge_identifiers(&mut self, other: &EnvProject) {

Conversation

victor-linroth-sensmetry commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrius-puksta-sensmetry commented Feb 11, 2026

Uh oh!

victor-linroth-sensmetry commented Feb 11, 2026

Uh oh!

daumantas-kavolis-sensmetry commented Feb 12, 2026

Uh oh!

daumantas-kavolis-sensmetry commented Feb 12, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrius-puksta-sensmetry Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrius-puksta-sensmetry commented Mar 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

victor-linroth-sensmetry commented Mar 25, 2026

Uh oh!

andrius-puksta-sensmetry commented Mar 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrius-puksta-sensmetry Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrius-puksta-sensmetry Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrius-puksta-sensmetry commented Mar 25, 2026

Uh oh!

victor-linroth-sensmetry commented Mar 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrius-puksta-sensmetry Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

victor-linroth-sensmetry commented Feb 11, 2026 •

edited

Loading

andrius-puksta-sensmetry Mar 25, 2026 •

edited

Loading

andrius-puksta-sensmetry Mar 25, 2026 •

edited

Loading

andrius-puksta-sensmetry Mar 25, 2026 •

edited

Loading

andrius-puksta-sensmetry Mar 30, 2026 •

edited

Loading