The missing CI for your ML supply chain

What it does

DataHub holds two graphs that no other catalog holds together: column-level lineage across the warehouse, and ML metadata for the models. Nothing joins them, so a model is not connected to a single column and a data failure cannot be traced to the model it breaks.

Janus writes that join (janus link) and then reads across it. Every check below exists because of that one edge: a feature knows the exact source column it came from, so a stale table, a leaking label or a classified column can be walked forward into the live model that consumes it, with the column chain as evidence.

Read right to left, that edge is what makes a verdict possible: from the live model back to the exact column, so a finding arrives as contract_renewed_flag <- churn rather than "something upstream of this model looks wrong".

Measured, not claimed

The same graph, the same ground truth, three ways of reading it, scored per feature: every approach can tell that a leaking model leaks, and the question that separates them is which of its features leaks, which is what somebody has to go and fix.

Precision at naming which feature leaks. Recall is 1.00, 1.00 and 0.00 respectively: the middle row does catch the leak, it just cannot say where.

Scored on a graph this project did not build

Everything above runs on the graph janus-seed wrote, which is the graph where the links the detectors read already exist. So there is a second run that removes the seeder entirely: examples/real-project/ is a postgres warehouse holding a public dataset, a dbt project, a scikit-learn training script and an MLflow registry, ingested by DataHub's own postgres, dbt and mlflow sources. The leak is written into the dbt model rather than planted by a seeding call, and the column lineage the walk follows is what DataHub's SQL parser produced from the compiled query.

Six of those seven features are clean, so a detector that answered "leak" to everything would score 0.14 here. The derivation it quotes (contract_renewed_flag <- churn) comes out of DataHub's own column-level lineage: no part of that path was written by this project. The section also states what it could not score on that stack, which is freshness, drift and the governance checks, because a real dbt project has no planted lag, schema change or classification in it.

Whether the benchmark could catch a broken detector at all

A precision table measures whether the detectors are right about a graph we chose. It says nothing about whether the trials would notice if a detector broke. That is a different question and it has its own answer: mutation testing, scoped to janus/detect/, because the claim under test is detection and mutating the rest of the package would measure something else.

Every survivor is listed with a verdict, grouped by root cause: a real gap in the trials, or provably equivalent with the reason written out. A survivor with no verdict fails the report rather than being published silently, so the number cannot be improved by looking away. The two recurring classes are named once rather than thirty times: a continue mutated to break inside a loop the fixture only ever gives one item, and a finding's own URN swapped for None where a trial checks that a finding exists without checking what it says.

And whether the prose invents figures

Narrative quality stays unscored on purpose: a readability rubric is soft evidence that varies by provider. Narrative faithfulness is checkable and is checked. Every figure in generated prose must appear in the facts the narrator was shown, and every URN in it must resolve in the graph. A model that divides a 30-hour lag by a 6-hour SLA and writes "five times" has produced a figure nobody measured, and a reader cannot tell it from one that was.

Measured at 1.00 for the template narrator, which is what CI has a credential for. That rate is worth reading only beside the count of figures it actually checked, which RESULTS.md prints next to it: prose quoting no number at all is faithful by this measure and says nothing. A provider row appears only when a key for it was present, and its absence is stated rather than left to look like a pass.

Method, ground truth and what is still not measured: benchmarks/RESULTS.md.

Install

Linux, Python 3.11 exactly. Nothing else is required for a local run: the DataHub Quickstart ships with metadata service authentication disabled, and no LLM key is needed.

pip install janus-datahub          # the CLI and the Python API

pip install "janus-datahub[agent]" # scan --review, the approval agent
pip install "janus-datahub[mcp]"   # the janus-mcp server
pip install "janus-datahub[pet]"   # Argos, the desktop window
pip install "janus-datahub[feast]" # link --from feast
pip install "janus-datahub[kafka]" # watch --events
pip install "janus-datahub[otel]"  # metrics to an OTLP collector
pip install "janus-datahub[openai]"    # or [anthropic], or [google]

Developing on Janus itself, or before the first release is cut:

git clone https://github.com/Ahmedxsaad/janus
cd DataHub
pip install -e ".[dev]"
cp .env.example .env     # DATAHUB_GMS_URL=http://localhost:8080

Quickstart: watch a failure fire and clear

This builds the ML supply chain the DataHub datapacks lack, plants one silent failure, and detects it end to end. About two CPUs and 8 GB free.

datahub docker quickstart            # UI at http://localhost:9002 (datahub/datahub)

janus-seed                      # the ML supply chain: tables, columns, model, lineage
janus-scenario --lag-hours 30   # a source table silently stops refreshing
janus scan --table loans_raw    # detect, explain, write back

The scan names the live model at risk, then writes the incident, the model-at-risk tag, the risk properties, the guarding assertion and the impact report into DataHub. Run it twice: nothing duplicates.

Then recover and rescan:

janus scan --table loans_raw --dry-run   # detect and explain, write nothing
janus-scenario --revert                  # the table refreshes
janus scan --table loans_raw             # no finding, no writes

inventory: what can already be checked

Your DataHub is not the demo graph, so start by asking what Janus can see. Expect most models to come back "not checked", and that is the honest answer rather than a failure: out of the box a model is not connected to a single column, so a detector that walks from a feature to its source column has nowhere to start.

janus inventory        # every model, and what can and cannot be checked

It also prints how long findings have been staying open. Mean and median time to resolution, per finding type, read straight out of incidentInfo.created and the resolution stamp GMS wrote on Janus's own incidents. Nothing new is recorded to make that possible, and the median sits beside the mean because one incident left open across a weekend moves a mean by days and describes none of the others.

Including the model versions search hides from you

Registering a second version of an MLflow model produces a second mlModel entity and stamps the first one's isLatest false. GMS then drops every non-latest version out of search results, so the earlier entity is invisible to any query that finds models by searching, while remaining perfectly alive: not deleted, still carrying its aspects, still carrying whatever Janus wrote to it.

Every sweep here (inventory, scan --all-models, link --all) turns that hiding off, because two of its consequences are silent correctness failures rather than cosmetic ones: a model link --all cannot see is one that quietly stops being checked after the next ingest, and an incident raised on a version that later stops being latest can never be resolved, because nothing reaches the model to notice the finding is gone. Scanning every version costs less than it sounds, since the detectors only write for a model somebody linked, and an unlinked old version reports itself unchecked and writes nothing.

Before anybody links anything: the table-level answer

Until a model is linked, none of the column-level checks can run on it. Rather than only listing what it could not do, a scan says what it can see about the tables that model is recorded as training on: whether one is past its freshness SLA, marked deprecated by its owners, or holds a column your organization classified. It is a distinct finding type, it never outranks a column-level finding, it contributes nothing to the trust score, and it says out loud what it cannot see:

Checked at table level only (churn_model declares no features): the table this model trains on is past its freshness SLA. Which of the model's features carry the stale values is not knowable without a column-level link. Asked which feature carries it, table-level reasoning scores a measured precision of 0.25 (benchmarks/RESULTS.md, table-level baseline), which is why this finding names the table and not a feature. Run janus link to get the column-level answer instead.

That 0.25 is measured, not asserted: it is the table-level baseline in the benchmark above, and the benchmark checks the figure the tool quotes against the one it measures on every run, so the disclosure cannot drift from the measurement in silence. It is a tool upselling its own accurate story with its own numbers rather than with a claim.

The same sweep as one number

inventory answers per model. coverage folds that sweep into the figure a platform lead reports upward, names the single next declaration that would raise it most, and with --write records the point on Janus's own dataFlow so the next sweep has a direction to compare against. It measures how much has been declared, not how healthy the models are: a catalog at 8% is one where Janus mostly cannot tell you either way.

janus coverage --write

The ranking is keyed on the remedy rather than on the check, because one link unblocks three checks at once and a ranking by check would recommend the same action three times. Freshness is deliberately not in the figure: it is asked of a table, and folding one table check and five model checks into a single percentage divides two different denominators.

And the one report for a budget holder

finops lists the tables whose every downstream model has no deployment in service and has gone untouched past JANUS_UNUSED_MODEL_DAYS. One live consumer and a table is not listed, because that is not a saving. A model whose catalog entry carries no date at all is reported separately as undated and never as unused: in the one report here that suggests deleting something, an absence is not evidence. It writes nothing and raises no incident, because nothing here is broken.

janus finops

link: join the model to its columns

janus link declares a model's features (one per column, each carrying the exact source column it came from), marks the label column with the glossary term the leakage detector reads, and captures the input schema as the baseline drift is measured against.

If your stack already declares it, do not type it twice

A Feast repo and a dbt semantic model already say which column each feature is read from, in a file your training pipeline reads and your team keeps correct. link imports it instead of asking:

janus link --model churn_model --from feast --repo ./feature_repo
janus link --model churn_model --from dbt   --repo ./churn_analytics

The readers are offline and read-only: they parse the declaration on disk and never connect to Feast, to dbt, or to a warehouse. Feast needs the package (pip install "janus-datahub[feast]"); dbt needs nothing at all, because a manifest is JSON, so this works against a manifest.json somebody sent you on a machine with no dbt.

Read from the feast declaration:
  read 'churn_model_v1' from the Feast repo at ./feature_repo
  feature table: warehouse.analytics.customer_features, the batch source of 'customer_features'
  features: 3 declared, of which 1 name a warehouse column different from the feature (from the source's field_mapping)
  label: churned of warehouse.analytics.customer_labels, from label view 'churn_label'
  not features: customer_id, event_timestamp (entity join keys and event timestamps), so they are excluded from the link

Features 'churn_model_v1' declares:
  tenure <- tenure_months  (feature view 'customer_features')
  monthly_charges <- monthly_charges  (feature view 'customer_features')
  support_calls <- support_calls  (feature view 'customer_features')

That first line is the case a name match gets wrong: the feature is tenure and the column is tenure_months, and only the declaration knows. Where a declaration is silent it says so rather than guessing (a dbt semantic model names no label, so --label-column stays yours to give), and where it names a column the table does not have, the import stops instead of linking the rest: a half-declared model is one whose unchecked columns nobody would ever hear about. --select picks between several declarations in one repo.

Or let it work the arguments out

janus link --model churn_model --infer

It tries four routes in descending order of confidence and tells you which one answered: the inputs the training run recorded (dataProcessInstanceInput), a run parameter naming a table (where DataHub's mlflow source puts MLflow params), a dataset the catalog already declares upstream of the model, and failing all of those, a shortlist of nearby tables for you to pick from.

Inferred from the graph:
  feature table: the only input recorded on churn_model's training run(s), from dataProcessInstanceInput
  label column: churned matches a known label name (JANUS_LABEL_COLUMN_NAMES). This one is a guess: check it
  excluded columns: customer_id, from the schema's own key declarations (primaryKeys, isPartOfKey, isPartitioningKey) and the label itself

Proposed:
janus link \
  --model churn_model \
  --features analytics.customer_features \
  --label-column churned \
  --exclude customer_id

Declare this? [Y/n]

There is no LLM in that, and nothing is written until you answer. Where nothing in the graph names a label at all it refuses to invent one and asks for --label-column, because a wrong label makes every leakage verdict wrong in both directions. Exclusions come only from the warehouse's own key declarations, never from column names that look like identifiers: customer_id is usually a join key and score_id is usually a feature, and no rule over names tells them apart.

Or type it

janus link \
  --model churn_model \
  --features analytics.customer_features \
  --label-table analytics.customer_labels \
  --label-column churned \
  --exclude customer_id

Keep the link alive

DataHub's mlflow source upserts the whole mlModelProperties aspect and drops the features. The arguments are recorded on the model in an aspect ingestion does not touch, so the replay needs no arguments at all and one command covers every model at once. A model nobody has linked is skipped rather than guessed at, so --all is safe on a schedule.

datahub ingest -c mlflow.yml     # your existing pipeline, unchanged
janus link --all            # put back what it dropped, for every linked model
janus scan --all-models     # audit the whole catalog

One line in the training script makes the next ingest self-describing, which is what lets --infer read the feature table rather than guess:

mlflow.log_param("janus_features", "analytics.customer_features")

Or stop replaying it by hand entirely: watch --events re-applies the link catalog-wide the moment an ingest drops it. That is further down the page, because it is the failure mode rather than the feature that is worth reading first.

scan: detect, explain, write back

janus scan --table loans_raw       # start from a table, walk into the models
janus scan --model churn_model     # start from a model, walk back to the columns
janus scan --all-models            # the whole catalog
janus scan --model churn_model --dry-run   # detect and explain, write nothing
janus scan --model churn_model --review    # pause, show, write only what you approve

What a scan writes back, all of it idempotent and keyed so reruns never duplicate:

The trust score, led by what is wrong with it

An integer out of 100 invites a reader to compare it against a threshold nobody calibrated. So the number comes last, after the deductions that produced it, each naming the finding that caused it:

100  starting
-20  leakage: Target leakage in credit_risk_v3.applicant_income
-15  freshness_lag: Stale upstream data in ecommerce.public.loans_raw
-10  missing_owner: nobody owns Credit Risk v3
---
 55  watch

Printed under it, every time the number is shown: the weights behind this score are a stated preference ordering, not a calibrated model. Compare scores to each other, not to a threshold. Every history entry carries a SCORING_VERSION, so a change to the weights reads as a labelled discontinuity in the trend rather than as a mystery drop, and a test fails if the weights move without the version moving with them.

The input contract, as a standard rather than as ours

An incident says what went wrong. A contract says what was expected to be true in the first place. --contract-out renders every table the model trains on as an Open Data Contract Standard file (ODCS v3.1.0, Linux Foundation Bitol): the schema each input should have and how fresh it should be, quoted from the graph's own schemaMetadata rather than invented. Anything that speaks ODCS can enforce it, not only Janus.

janus scan --model credit_risk_v3 --contract-out ./credit_risk_v3.odcs.yaml

The six checks, and what each one needs

A scan never reports something healthy that it could not measure. It names the check, the missing metadata, and how to supply it.

The three that read the governance graph

The first three checks ask whether a model's data is correct. The last three ask something the organization has already answered elsewhere in DataHub, and that nothing today joins back to the model.

Sensitive source. Somebody classified a column as PII, PHI or restricted. Three joins downstream, a feature derives from it and a live model trains on that feature. Nothing is broken; what is wrong is what the model was allowed to see. It is the leakage walk with a different mark, so it produces the same auditable proof:

credit_risk_v3 feature applicant_income derives, through applicant_income <- income, from loans_raw.income, classified janus.sensitive.

JANUS_SENSITIVE_TAG_URNS=urn:li:tag:PII,urn:li:tag:Confidential
JANUS_SENSITIVE_TERM_URNS=urn:li:glossaryTerm:Classification.Restricted

There is deliberately no default. A guessed classification URN either matches nothing or matches a term that means something else in your catalog, and a false incident about a compliance exposure is the worst kind to be wrong about. Leave both empty and every scan reports the check as not evaluated, never as clean.

Deprecated input. A table's owners marked it deprecated, with a note and sometimes a decommission date. They have no way to know a model depends on it. This needs no configuration: deprecation is DataHub's own aspect with one meaning everywhere. It is never more than medium severity, because it is a deadline rather than a defect.

Proxy candidate. A feature and a column somebody classified as a protected attribute both descend from a common ancestor, with neither descending from the other. That shape is how a proxy usually arises, and it is also how two harmless siblings arise, so the finding is raised as a question for a human and capped at medium forever: it never escalates on a live model, it contributes nothing to the trust score, and its first suggested remedy is to review it, which no machine here will perform for you. Whether a feature actually proxies for a protected attribute is a statistical claim about data this tool has never read a row of. What it can say, structurally, is where to look, and it can say it before the model is trained. Configured, like the classification above, from your own taxonomy with no default:

JANUS_PROTECTED_ATTRIBUTE_TAG_URNS=urn:li:tag:ProtectedAttribute
JANUS_PROTECTED_ATTRIBUTE_TERM_URNS=urn:li:glossaryTerm:Protected.Ethnicity

A direct descent is P5's finding and is excluded here rather than reported twice: a proved derivation is not a candidate. The first two checks are reversible scenarios, so you can watch them fire and clear:

janus-scenario --scenario sensitive-source
janus scan --model credit_risk_v3
janus-scenario --scenario sensitive-source --revert

The fix, not just the fault

Every finding carries a counterfactual: a set of changes, each one sufficient on its own to clear it. Not advice, and not a language model's suggestion. It is derived from the same traversal that found the problem, which already computed every derivation path and previously threw all but the shortest away.

Target leakage in credit_risk_v3.applicant_income  (severity: critical)
  feature      applicant_income
  leak path    applicant_income <- income_band <- default_flag
  label        loans_labelled.default_flag
  model        Credit Risk v3 LIVE

  Any one of these clears this finding:
    - Rebuild applicant_income from data known before the outcome is, so it
      stops deriving from the label: cut applicant_income <- income_band,
      applicant_income <- default_flag_backfill.
    - Drop the feature applicant_income from Credit Risk v3 and retrain, so
      nothing the model reads carries the answer.
    - If loans_labelled.default_flag is not this model's label, remove the
      label declaration from it: the finding rests on that declaration and is
      wrong without it.

  This finding rests on 2 distinct derivation paths. Cutting one of them
  leaves it standing: the cut listed above names the first edge of every
  path, and all of them have to go.

That last clause is the moat stated inside an incident somebody actually reads. A table-level tool cannot express it, because it never saw the column edge in the first place. Each detector has its own set: a freshness finding clears when the table refreshes inside its SLA or the model stops consuming it, a drift finding when the column returns to its training-time type or the model is retrained, a sensitive-source finding when the derivation is cut or the classification is corrected (worded as a correction and never as a dismissal, naming the owner of the classification), and a deprecated-input finding when the model moves to the successor the deprecation note names. If there is no note, it says to go and ask rather than inventing a table name.

Each one was applied, and the detector asked again

A suggested fix nobody performed is not a measurement. So the benchmark performs them: it applies each counterfactual to the live graph and reruns the same detector.

The last column is not a gap being hidden. Retraining a model, migrating onto a successor table and dropping a feature are real fixes that no metadata write can carry out, so they are named as unverified rather than counted as passes.

This is the counterfactual explanation of the xAI literature (Wachter, Mittelstadt and Russell, 2017) applied to a lineage graph rather than to a feature vector. It is fully deterministic and needs no model, no rows and no LLM: the same walk, asked what would have to be different.

gate: block a bad model before it merges

Everything above audits a graph that already holds the mistake. janus gate is the preventive half, for a pull request. It runs the same detectors, judges them against a policy, and answers in an exit code.

janus gate --model credit_risk_v3 --block-at-or-above high   # exit 1 if it leaks
janus gate --model credit_risk_v3 --min-trust 80             # exit 1 if trust < 80

The third one is the point. A gate that reported "I could not connect" as a policy violation would teach a team to wave through every red build, so a setup failure never masquerades as a finding. The gate writes nothing by default, because it runs on every push and one incident per run would fill the graph with findings about branches that never merged.

--min-trust on its own prints a warning, because it is the blunt secondary control: a score is a preference ordering and a threshold over it is not a policy. Pair it with --block-at-or-above, which judges the findings themselves.

In a workflow

- uses: Ahmedxsaad/janus@main
  with:
    model: credit_risk_v3
    block-at-or-above: high
    gms-url: ${{ secrets.DATAHUB_GMS_URL }}
    gms-token: ${{ secrets.DATAHUB_GMS_TOKEN }}

The verdict lands on the run's own summary page, not just in the log: findings, severities, trust scores and the checks that could not run, as a table the reviewer sees without opening anything. That needs no input and no token, because GitHub already gives every step a GITHUB_STEP_SUMMARY file to append markdown to; outside Actions the variable is unset and nothing is written.

watch: keep looking

watch is the long-running entry point: it polls, scans on change, and auto-approves its writes because it is unattended by definition. It is what the janus-watch Helm chart runs, alongside an optional link --all CronJob for the models that are not retrained nightly.

janus watch --table loans_raw           # poll one table
janus watch --table loans_raw --pet     # and put Argos on the desktop

Its own numbers, where your other numbers are

Every completed scan already logs what an SLO is built from: findings raised, writes made, and how long detection itself took, separate from the poll interval and from DataHub's own indexing, because watch controls neither. Set JANUS_OTEL_ENDPOINT and the same three numbers go to an OTLP collector as metrics. Unset, and nothing is imported and nothing is exported.

Three instruments and no traces, on purpose. A team that wants spans across the DataHub SDK's HTTP calls installs opentelemetry-instrumentation-requests and gets them, which is better than this project shipping a second, worse copy of it.

pip install "janus-datahub[otel]"
JANUS_OTEL_ENDPOINT=http://localhost:4318/v1/metrics janus watch --table loans_raw

React to the graph instead of a timer

watch --events consumes DataHub's own MetadataChangeLog instead of polling, and does one thing polling structurally cannot: it re-applies, catalog-wide, any janus link an ingestion run drops.

It replays only what a human already confirmed, so a model nobody linked is left alone: an inferred join looks identical to a confirmed one in the graph and would make every detector downstream confident about the wrong columns. Polling remains the default and needs no broker.

pip install "janus-datahub[kafka]"
janus watch --events --model credit_risk_v3

Call it from your training script

The command line is the main interface, but there is one place Janus belongs inside your code: the script that trains the model. That is the only moment when the feature table, the label column and the training-time schema are all known.

import mlflow

from janus import link_model, scan_model

FEATURE_TABLE = "analytics.customer_features"

# Logged as a run parameter as well as declared: the parameter survives into
# DataHub through the ordinary mlflow ingest, which is what lets `link --infer`
# read the table next time instead of guessing at it.
mlflow.log_param("janus_features", FEATURE_TABLE)

link_model(
    model="churn_model",
    features=FEATURE_TABLE,
    label_column="churned",
    exclude=["customer_id"],
)

report = scan_model(model="churn_model", dry_run=True)
if not report.clean:
    raise SystemExit(f"{len(report.writes)} finding(s) before this model ships")

Declared here, the link is re-declared by the same run that produces the model, so the next training run repairs whatever the last ingest dropped.

Two functions and their result types, and deliberately no more: those names are the supported surface a script may pin to. They are thin wrappers over exactly the functions janus link and janus scan call, so a finding found here is found identically at the command line. Both read .env the same way the CLI does; pass conn= to reuse one connection across many models. Everything else in the package is importable and documented, but its shape is free to change.

JSON output

Routing findings somewhere Janus does not know about? Both scan and gate take --format json and put the whole report (evidence, models at risk, trust deductions, each finding's counterfactual, the gate's violations) on stdout as one parseable document, with progress lines moved to stderr so the stream stays clean. The counterfactual goes out as data rather than as prose: a stable remedy kind and its targets, so a consumer can act on it without parsing an English sentence.

janus scan --model credit_risk_v3 --format json | jq '.findings[].severity'
janus scan --model credit_risk_v3 --format json | jq '.findings[].counterfactual.remedies[].kind'

Ask it, do not type it

pip install "janus-datahub[mcp]"
janus-mcp   # serves check_leakage, check_freshness, check_gate over stdio

Point an MCP client at the installed janus-mcp command and ask "is credit_risk_v3 leaking?" in plain language. All three tools are read-only, enforced at registration (readOnlyHint: true) and by calling every scan in dry-run with no way to turn that off: the model on the other end of an MCP client is not Janus's own narrator, it is outside this project's control entirely, so it gets to ask what is wrong and nothing more.

It is meant to run beside DataHub's own mcp-server-datahub, not instead of it. That server answers what the catalog contains: search, lineage, schemas, ownership, the open-ended questions where a model's job is to explore. Janus answers the three that have to be reproducible, with the column chain as evidence and no LLM anywhere in the decision.

Argos, the pet on your desktop

pip install "janus-datahub[pet]"      # macOS and Windows; Linux: the .deb or
                                           # .AppImage on the GitHub release
janus watch --table loans_raw --pet   # Janus's own findings
janus companion                       # everything wrong with the assets you own

Argos is a 32x32 pixel watchdog that sits on your desktop and shows what the graph is doing. He patrols while a poll finds nothing, sniffs while a lineage walk is in flight, barks with a red collar the moment a finding lands, and turns into a translucent ghost when he cannot reach DataHub, because a cheerful pet on a disconnected watch is the lie that gets ambient status displays switched off. Nothing he does is on a timer: every state is an event a detector actually produced.

Double-click a finding and he walks the blast radius across the screen, one hop per graph hop, with the column name floating over each jump. That is the column-level traversal the benchmark above measures, rendered as motion instead of a paragraph.

janus companion is the half that is not about Janus at all: it runs no detector, and sweeps the assets one owner owns for open incidents, failing assertion runs and deprecations. DataHub has no desktop presence today, and that is the gap it fills.

With no window binary installed, both commands report one line per change in the terminal instead, which is also what runs over SSH. Design and protocol: docs/plan/08-watchdog-mascot.md.

Run it without a Python install

datahub docker quickstart              # once: builds DataHub's own stack
docker compose run --rm janus-seed
docker compose run --rm janus scan --table loans_raw
docker compose run --rm janus gate --model credit_risk_v3 --block-at-or-above high
docker compose up janus-mcp       # long-running, stdio

The compose file adds Janus to the Docker network datahub docker quickstart already creates, rather than reimplementing DataHub's own multi-container stack inside this repository.

Every flag, in one table

The sections above show each command doing the thing it exists for. This is the whole of the rest, so no option is reachable only by typing --help. Every command also takes --help, and none of them take a positional argument.

scan

The two -out flags are for the reader who has no DataHub login: the same report and the same assertion the scan writes into the graph, as files to attach to a ticket or commit beside the model. They are additional, never instead of the write.

gate

link

watch

companion

The read-only reports

--limit is there because a first run against somebody else's catalog should be cheap to try. It is a stop, not a sample: the models are taken in the order the graph returns them, so a limited sweep answers honestly about the models it looked at and says nothing about the rest.

The other three commands

janus-seed         # build the demo ML supply chain
janus-scenario     # plant a failure: --lag-hours, --scenario, --revert
janus-mcp          # the read-only MCP server, over stdio

Configuration

Configuration enters the process in exactly one module, janus/env.py, and comes from .env. Copy .env.example and fill it in; the two files carry an identical key set in the same order, so copying one produces a working run.

DATAHUB_GMS_URL=http://localhost:8080
DATAHUB_GMS_TOKEN=                      # not needed for a local Quickstart

JANUS_LLM_PROVIDER=                # anthropic, openai or google. Optional
JANUS_LLM_MODEL=
JANUS_LLM_API_KEY=

JANUS_SENSITIVE_TAG_URNS=          # your own classification taxonomy
JANUS_SENSITIVE_TERM_URNS=
JANUS_PROTECTED_ATTRIBUTE_TAG_URNS=    # and your own protected attributes
JANUS_PROTECTED_ATTRIBUTE_TERM_URNS=
JANUS_LABEL_TERM_URN=              # honor your existing label term
JANUS_OTEL_ENDPOINT=               # OTLP metrics, off unless set

Three rules govern that file, and they are worth knowing before you debug a run:

• Anything that identifies a system, an account or a vendor has no default and no fallback. Server URLs, tokens, API keys, provider names, model ids. A fallback is a machine-specific value in tracked code: it turns a missing .env into a silent connection to the wrong place, or a silent call to the wrong vendor billed to whatever key is in the ambient environment. Missing means missing, and it fails loudly, naming the variable.
• A group of related settings is all-or-nothing. Set every one or none. A half-configured feature fails loudly, it never downgrades in silence.
• Algorithm parameters are not identity. Thresholds, hop caps and score weights keep documented defaults in janus/config.py, and you can override them.

Already have a glossary term for labels? Point JANUS_LABEL_TERM_URN at it and the detector honors yours instead of creating one.

The parameters, with the defaults they ship with

These are the third rule above: they change how hard the detectors look, not what they connect to, so each carries a documented default and a run with an empty .env uses it.

JANUS_FRESHNESS_SLA_HOURS=6      # past this, a table is stale
JANUS_MAX_HOPS=3                 # how far a blast-radius walk goes downstream
JANUS_LEAKAGE_MAX_HOPS=6         # a leak path may be longer than a blast radius
JANUS_PROXY_MAX_HOPS=3           # how far up to look for a common ancestor
JANUS_LINEAGE_RESULT_CAP=500     # edges read per hop, so one hub table cannot stall a scan
JANUS_LABEL_COLUMN_NAMES=        # the names --infer will propose as a label
JANUS_UNUSED_MODEL_DAYS=90       # idle window before finops calls a model unused
JANUS_COMPANION_ENTITY_CAP=200   # assets one companion sweep reads

The two hop caps are separate on purpose. A blast radius answers "who is affected", and three hops of that is already a wide answer. A leak path answers "where did this value come from", and a real warehouse puts six joins between a label and the feature that quietly carries it, so the same cap on both would either miss leaks or make every scan expensive.

And the operational half

JANUS_LOG_FORMAT=                # json, for a log pipeline. Unset: human-readable lines
JANUS_OTEL_HEADERS=              # for a collector behind an auth header

JANUS_KAFKA_BOOTSTRAP=           # the three watch --events needs, all or none
JANUS_SCHEMA_REGISTRY_URL=
JANUS_KAFKA_GROUP_ID=

JANUS_ARGOS_BIN=                 # the desktop window, if it is not on PATH
JANUS_DATAHUB_UI_URL=            # the UI port, not GMS: what Argos opens on a click
JANUS_COMPANION_OWNER=           # whose assets janus companion sweeps

JANUS_LOG_FORMAT=json is for the one process that runs for days. A human tailing watch wants scan complete run_id=... findings=1; Loki or an ELK stack wants fields it can index without a per-tool regular expression that breaks the first time a key is added. Both come from the same call site, so a field cannot exist in one and be missing from the other.

JANUS_DATAHUB_UI_URL is a different port from DATAHUB_GMS_URL above (9002 rather than 8080 on a Quickstart), which is why it is asked for rather than derived. Without it Argos says so on the finding instead of opening a browser at a guess. The Kafka three are a group, so setting one of them and not the others fails loudly rather than quietly falling back to polling.

Security model

• The LLM never decides anything. Detection is deterministic Python. Catalog text (descriptions, column names, glossary definitions) is attacker-controllable, so it is wrapped as delimited untrusted data and delimiter lookalikes are stripped before wrapping. Even a successful injection cannot invent a finding: it is downstream of the detectors.
• No row of your data ever leaves DataHub. Every check above is computed from metadata: lineage edges, schemas, timestamps, tags and terms. There is no warehouse connection to configure, so there is none to leak, and the narrator is shown facts rather than samples.
• Writes are fixed and parameterized. A closed set of mutations with validated arguments. There is no code path that sends a GraphQL string the caller supplied.
• Writes are idempotent and reversible in place. Keyed on (resource_urn, incident_type, title) with read-before-write, stamped with a janus.run_id for provenance. The benchmark reads the graph back after a rerun and measures the duplicates created: 0.
• Writes are gated on a human. scan --review writes only what you approve. gate does not write unless you pass --write. The MCP tools cannot write at all, on any flag. watch auto-approves because it is unattended by definition.
• The token stays a secret. It enters the process in one module, lives only in a git-ignored .env, and is never logged, echoed or put in an exception message. Errors name the variable, never its value.
• Least privilege, honestly. DataHub OSS personal access tokens are not scoped per operation, so Janus cannot claim a narrowed token. What it can say is what it touches: incidents, tags, glossary terms, structured properties, documents and assertion aspects. Give it a token you are willing to rotate, and rotate it.

Each detector implements a published result rather than a heuristic somebody liked:

Janus inside the graph it guards

The product's thesis is that a pipeline nobody catalogued is a pipeline nobody can reason about. An agent that writes incidents into a graph while remaining invisible in that same graph is exempting itself from its own argument. So every scan is emitted as what it is: a process run.

No new dependency, no scrape endpoint and no second system to run: those are DataHub's own shipped entities. What it buys is that an incident stamped Raised by Janus run scan-cb2ef85ba9a8 stops sending a reader to grep a log file. The run is an entity they can open, whose inputs say what was examined and whose outputs say what was touched.

A scan that dies emits a failed run rather than silence. Detection runs inside the run too, so a scan that dies deciding what is wrong is recorded the same way as one that dies writing it down. That is the difference between "clean" and "never completed", and without it a half-written graph says nothing at all. A --dry-run emits nothing, and a rerun with the same run_id converges rather than stacking, like every other write here.

Where this fits a governance framework

This is a mapping, not a conformity claim. Each row says which NIST AI RMF subcategory a detector's output is evidence for; whether that subcategory is satisfied is a judgement about your organization's whole process, which no tool reading a metadata graph can make. Subcategory text is quoted from the NIST AI RMF 1.0 Playbook.

Generated from the detector registry rather than typed by hand, so a check cannot be added to Janus without appearing here. Run janus crosswalk to print this table as markdown, straight into whatever document you have to file.

The subcategories cited above, quoted from the NIST AI RMF 1.0 Playbook:

• MAP 2.3 Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection.
• MAP 4.1 Approaches for mapping AI technology and legal risks of its components, including the use of third-party data or software, are in place, followed, and documented.
• MAP 4.2 Internal risk controls for components of the AI system including third-party AI technologies are identified and documented.
• MEASURE 2.3 AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s).
• MEASURE 2.4 The functionality and behavior of the AI system and its components are monitored when in production.
• MEASURE 2.5 The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the conditions under which the technology was developed are documented.
• MEASURE 2.10 Privacy risk of the AI system is examined and documented.
• MEASURE 2.11 Fairness and bias, as identified in the MAP function, are evaluated and results are documented.
• MANAGE 2.2 Mechanisms are in place and applied to sustain the value of deployed AI systems.
• MANAGE 2.3 Procedures are followed to respond to and recover from a previously unknown risk when it is identified.
• MANAGE 4.1 Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors, appeal and override, decommissioning, incident response, recovery, and change management.

Three documents, generated instead of maintained

janus model-card    --model credit_risk_v3   # prints; --write publishes it to DataHub
janus evidence-pack --model credit_risk_v3   # EU AI Act Article 10
janus feature-card  --model credit_risk_v3   # one Data Card per feature

A model card in the sense Mitchell et al. (FAT* 2019) proposed one: intended use, which columns each feature was actually computed from, the reported training metrics, the trust score with its waterfall, the findings open against the model, and the checks that could not run. Intended use is the one section a graph cannot derive, so an undeclared one says exactly that rather than being omitted. An evidence pack mapping the same graph to Regulation (EU) 2024/1689 Article 10 and Article 12, by paragraph number (10(2)(b), 10(3), 10(2)(f), 10(5)), so a reader can check the mapping rather than trust it.

Neither is maintained by hand, which is the whole point: a hand-written model card is accurate until the model next changes, and most of them are already wrong. These are regenerated from the graph, so they are current by construction and empty where the catalog is. Anything DataHub does not record prints as not recorded in the catalog rather than being quietly dropped, so a gap is visible in the document instead of reading as an absence of a problem.

feature-card is the third, a Data Card in the sense Pushkarna, Zaldivar and Kjartansson (FAccT 2022) proposed one, but for a single feature: where it is computed from hop by hop, every other derivation the walk found, each table that chain crosses and how current it is, whether the chain reaches a column classified as restricted or as a protected attribute, whether its type has moved since training, and, when a finding names it, the changes that would clear it. Taken per model because that is what somebody has in hand; --feature filters within it. Its freshness figures say out loud that they are measured now and not at training time, for the same reason the evidence pack refuses to substitute one for the other.

The evidence pack's first heading is This is not a compliance certification, and its second is What this pack could NOT establish: deliberately the first section rather than a closing caveat, because a gap at the end of a long document is a gap nobody reads. It states, for instance, that freshness at training time is unknowable from this graph (Janus measures freshness now, which is a different claim and not a substitute), and that whether anyone examined the data for bias is an activity no catalog records. A generated document that implied conformity would be worse than no document at all.

All three print by default and write nothing. --write publishes the document against the model, keyed on the model alone, so regenerating replaces it rather than leaving a second copy behind. Either way the underlying scan is read-only: asking for documentation never raises an incident as a side effect.

Everything else