AWS SSI: Enable profiling scenarios on the CI

[robertomonteromiguel]

Motivation

Changes

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core

[datadog-prod-us1-5]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 38 Tests failed

tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM.test_mix_snapshot[uds] from system_tests_suite     (Fix with Cursor)

AssertionError: assert 'Snapshot was not received' is None
 +  where 'Snapshot was not received' = <built-in method join of str object at 0x7fcbfb22b720>(['Snapshot was not received'])
 +    where <built-in method join of str object at 0x7fcbfb22b720> = '\n'.join
 +    and   ['Snapshot was not received'] = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7fcbf12bbaa0>.setup_failures

self = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7fcbf12bbaa0>

    def test_mix_snapshot(self):
>       self._assert()

...

tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Metric_Basic.test_ffe_eval_metric_basic[chi] from system_tests_suite     (Fix with Cursor)

AssertionError: Expected at least one feature_flag.evaluations metric for flag 'eval-metric-basic-flag', but found none. All eval metrics: []
assert 0 > 0
 +  where 0 = len([])

self = <tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Metric_Basic object at 0x7fcdccad4aa0>

    def test_ffe_eval_metric_basic(self):
        """Test that flag evaluation produces a metric with correct tags."""
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
    
...

tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Metric_Basic.test_ffe_eval_metric_basic[echo] from system_tests_suite     (Fix with Cursor)

AssertionError: Expected at least one feature_flag.evaluations metric for flag 'eval-metric-basic-flag', but found none. All eval metrics: []
assert 0 > 0
 +  where 0 = len([])

self = <tests.ffe.test_flag_eval_metrics.Test_FFE_Eval_Metric_Basic object at 0x7f370eab9370>

    def test_ffe_eval_metric_basic(self):
        """Test that flag evaluation produces a metric with correct tags."""
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
    
...

View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ad8b81b | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}