CHESSExperiments API Reference

LAYOUT_COLUMNS

The required layout DataFrame columns. well,row,col,run,positive,negative,run_index are derived from a scheduled PlateMaps.PlateMap/RunMaps.RunMap pair (see the CHESSExperimentsRunMapsExt extension); labware,location_id,metadata are CHESSExperiments' own additions.

RESERVED_DESIGN_COLUMNS

Design columns recognized directly by the scheduling machinery, never resolved as a reagent or a registered Factor – the same "fixed, recognized schema" status LAYOUT_COLUMNS has for :layout.

• :control_role: missing/absent = an ordinary sample row; a Symbol (:positive/:negative/ :blank/open-ended) = this row is a control template (see expand_control_templates).
• :duplicates: missing/1 = one physical well (no extra copies); N = N total physical wells for this row (the row's own well plus N-1 extra duplicates), aggregated back into one data point downstream by CHESSProcessing. Applies to any row, sample or control alike.
• :_block_key: internal – set by expand_control_templates on an expanded control-template copy to record which blocking partition it's meant to join, when that differs from what its own (deliberately fixed) column values would otherwise group it into. missing/absent on every ordinary row. See expand_control_templates/partition_by_blocking.

const factor_registry::Dict{Symbol,Factor}

Every registered Factor, keyed by name. Add to it with register_factor! and look factors up with get_factor rather than using it directly.

const parameter_registry::Dict{Symbol,ParameterKind}

Every registered ParameterKind, keyed by name. Add to it with register_parameter! rather than directly.

struct CategoricalFactor <: Factor

• levels: a stored zero-argument closure returning the currently-valid set of values – e.g. () -> (:aerobic, :anaerobic) for a closed, static vocabulary, or () -> keys(some_registry) for an open, growing one (organism, backed by CHESSLabConstants). One field/type serves both cases; the difference is entirely in what the closure captures.

struct ContinuousFactor <: Factor

A numeric factor – most commonly a reagent quantity (destination = :reagent), resolved via resolve_stock rather than a stored resolve function (see that docstring for why reagent factors need no per-factor registration at all).

struct DesignColumnMap

Per-input, unregistered translation for one parsed design's raw strings – deliberately not a durable global alias registry, since spreadsheet naming inconsistency ("Ala"/"L-Alanine"/ "alanine") is lab-specific/one-off enough that a growing global table would risk collisions rather than convergence. Supplied per parse_design call and stored as Experiment metadata (under :column_map) so a stored design's raw-string interpretation stays auditable.

Fields:

• columns::Dict{String,Symbol}: raw spreadsheet header -> canonical factor name. A header not present as a key is assumed to already equal its canonical name.
• values::Dict{Symbol,Dict{String,Symbol}}: canonical factor name -> (raw value -> canonical value). Only matters for factors whose values are themselves free text needing translation (open-vocabulary categorical factors like organism/strain) – not for numeric reagent quantities.
• units::Dict{Symbol,String}: canonical factor name -> unit string, one entry per :reagent- or :organism-destination factor (confirmed sufficient: one unit per factor, never per-row/per-cell). Feeds the unit half of what resolve_stock needs to convert a raw value into a real quantity (a Biomass for :organism factors).

DuplicateRegistrationError

Raised when something is registered under a name that's already taken (see register_qc_method!).

struct Experiment

The core fixture of CHESSExperiments. The governing rule for what belongs here versus in metadata: design is the plan – the treatments under study. metadata is everything about identifying, scheduling/executing, or processing that plan, not the plan itself.

Fields:

• design: the design matrix itself – one row per planned trial, one column per factor. Replication is structural: a treatment run 3x is just the same row appearing 3 times, not a separate field. Controls are deliberately not rows here – an experimenter specifies how many controls they want (see register_parameter!/get_parameter), not additional design-matrix entries. This is the one field that can't collapse into metadata: it doesn't describe or annotate the plan, it is the plan.
• metadata: everything else – name, layout (once scheduled), control counts, which QC methods apply, protocol notes, parsed/processed data once instruments produce it, whatever else describes, schedules, executes, or processes the design without being part of the design itself. See register_parameter!/get_parameter for typed, defaulted, validated access to specific keys here without Experiment needing to know about them in advance; unregistered keys are still just a raw Dict lookup. See with_parameter for setting a key.

Things that are already fully recoverable from design (e.g. "factors" = names(design)) don't get a field or a registered parameter – a plain function suffices, no storage needed either side.

A variable holding a whole Experiment should be named experiment (or expt), never design – that name is specifically the DataFrame field (experiment.design).

abstract type Factor

A named, registrable description of a design-matrix column (a "factor"). Two concrete subtypes distinguish domain shape – CategoricalFactor (a fixed or registry-backed set of levels) and ContinuousFactor (a numeric quantity) – independent of destination, which answers a different question: where a resolved value ends up.

destination values in use:

• :reagent (ContinuousFactor only) – feeds Stock-building (see resolve_stock).
• :organism – like :reagent, but for organisms: the column name identifies the organism/strain (resolved structurally via orgparse, mirroring how a reagent column is recognized), and the cell value is a Biomass quantity (paired with a per-column unit) fed into resolve_stock, promoting the row's Stock to a Culture. Also recorded per-well (:well_conditions) as a technician-facing readout independent of scheduling. Not a CategoricalFactor/ContinuousFactor destination – like :reagent, organism columns need no per-column Factor registration at all (see is_registered_organism).
• :condition (either subtype) – a plain recorded value (e.g. temperature, atmosphere), never a CHESSCore.Attribute – recorded per-plate (:plate_conditions).

destination is deliberately just a Symbol, not a closed enum – new destinations can be added without a package release, the same way unregistered metadata keys work for ParameterKind.

blocking is orthogonal to both the type (Categorical/Continuous) and destination – a temperature factor and an atmosphere factor can both be blocking despite differing on both other axes. Two design rows differing in any blocking factor's value must never be scheduled onto the same physical plate.

struct ParameterKind

A named, typed, registrable key into an Experiment's free-form metadata – mirrors QCMethod's registry (itself mirroring CHESSCore.Attribute/Read/LocationKind kind-registries). Lets a package (RunMaps, CHESSQC, ...) require e.g. :positive_controls::Int with a default and a validator, without Experiment needing to know about that key in advance. Any metadata key that isn't registered is still just a raw Dict lookup – registration is opt-in exactness, not a schema the whole metadata dict must conform to.

Fields:

• name: the key into metadata this kind governs.
• type: the expected Julia type of metadata[name], checked only when the key is actually present – default is returned as-is when absent, with no type check against it (so e.g. a DataFrame-typed parameter can still default to nothing, meaning "not set yet").
• default: value returned by get_parameter when name is absent from metadata; missing means the parameter is required (absence is an error).
• validator: optional value -> Bool predicate checked (only against a present value) after the type check; nothing skips it.

abstract type QCMethod

A named, registrable QC/correction method (e.g. CHESSQC's GP-based spatial correction). Concrete subtypes are defined by whichever package implements the method (CHESSQC and friends); this package only owns the registry, mirroring CHESSCore.register_lab/@attribute/@read's pattern of a global, name-keyed registry that any package can add to without the defining package needing to know about it in advance. This is the same split as CHESSCore (generic kind registries) vs. CHESSLabConstants (concrete registered kinds, plus the heavier domain-specific dependencies) – CHESSExperiments stays agnostic and dependency-light; the actual algorithms, and packages like GaussianProcesses/Optim they need, live in the package that implements them.

The registry is deliberately kept generic enough (name => type) that a future registry of entire named experiment workflows (e.g. "kinetic growth assay" — a design template + QC pipeline + expected data shape bundled together) can reuse the same mechanism once there's real workflow experience to draw the abstraction from; that type doesn't exist yet.

_blocking_columns(design) -> Vector{Symbol}

Every registered blocking Factor present as a column in design, in a fixed sorted order. The single source of truth for that order – both expand_control_templates (which builds :_block_key tuples in this order) and partition_by_blocking (which builds its group keys in this order) must agree, or a control's tagged key and a sample row's freshly-computed key could represent the same combination with values in different positions and silently fail to match.

_reagent_contribution(qty::Unitful.Quantity, reagent::CHESSCore.Reagent, total_volume)
    -> (contribution::CHESSCore.Stock, consumed_volume::Union{Unitful.Volume,Nothing})

Turn one reagent factor's raw (value, unit) quantity into a Stock contribution, using CHESSCore.Stock's own quantity-construction operators (Unitful.Mass/Volume/Amount * Reagent) rather than vc_to_stock/q_to_stock – those DataFrame-interop functions reject molarity outright, while the underlying Stock operators already convert a molar amount via molecular_weight. Also returns how much of total_volume this reagent consumed, if it's a liquid given as an absolute volume – used by resolve_stock's solvent shortcut.

classify_columns(names::Vector{Symbol}; reagent_context=CHESSCore, org_context=CHESSCore)
    -> (reagent=Vector{Symbol}, organism=Vector{Symbol}, condition=Vector{Symbol})

Group a design's (already-canonicalized) column names by factor_destination. RESERVED_DESIGN_COLUMNS are recognized (don't error) but land in none of these three lists.

control_role(design, i) -> Union{Missing,Symbol}

The :control_role value for design row i, or missing if the column isn't present at all (an ordinary sample-only design). missing/absent = an ordinary sample row.

destination(f::Factor) -> Symbol

Where a resolved value for f ends up – :reagent or :condition (:organism columns are resolved structurally, like reagent columns, and never go through a registered Factor).

expand_control_templates(experiment::Experiment) -> Experiment

Materialize every control template row (a row with a non-missing :control_role) into one concrete row per distinct blocking-column combination that actually occurs among the design's sample rows (never a full cross-product of each blocking column's values independently – only combinations that are actually tested). A template's own missing cells (its wildcards, excluding RESERVED_DESIGN_COLUMNS) get filled in from each combination; a cell the template already gave an explicit value stays exactly as written, even across different combinations – this is what lets a control's own composition (e.g. "no carbon source") stay fixed while it still gets one replicate per real block. A template with no wildcard cells at all is kept exactly as written (a single, fully-concrete control, no :_block_key needed). Every ordinary sample row is validated along the way: a missing value in a blocking factor column is only legitimate on a control template (the wildcard signal) – on a sample row it's an error, since a sample must specify every blocking factor's concrete value.

Every expanded copy is tagged with a reserved :_block_key column recording which blocking combination it's meant to join – read by partition_by_blocking instead of the row's own column values, since (as above) those can deliberately disagree with the block it belongs to. Runs before partition_by_blocking in the scheduling pipeline.

Narrowed from an earlier, more general design: wildcard traversal is scoped to registered blocking columns specifically (not any arbitrary missing column) – needed to make :_block_key well-defined.

factor_destination(name::Symbol; reagent_context=CHESSCore, org_context=CHESSCore) -> Symbol

Resolution order for a canonical column name: is it one of RESERVED_DESIGN_COLUMNS (returns :reserved)? else is it a reagent CHESSCore already knows (:reagent)? else is it an organism CHESSCore already knows (:organism)? else is it a registered Factor? Throws ArgumentError otherwise – an unregistered column is a hard error, a deliberate departure from ParameterKind's lenient "unregistered keys are just a raw lookup" stance, since a design has to be fully understood before it's usable.

factor_levels(f::CategoricalFactor) -> Any

Call f's stored levels closure to get the currently-valid set of values. Named factor_levels, not levels, to avoid colliding with DataFrames/DataAPI/Missings' own exported levels (categorical-array levels) – both would otherwise be ambiguous under a single using.

get_factor(name::Symbol) -> Factor

Look up the registered Factor for name. Throws KeyError if name isn't registered.

get_parameter(experiment::Experiment, name::Symbol)

Look up experiment.metadata[name]. If present, type-checked against the registered ParameterKind for name and validated if a validator was registered. If absent, returns kind.default as-is (no type/validator check against the default – it's allowed to be a sentinel like nothing that isn't an instance of type), or errors if the parameter is required (default === missing).

is_blocking(f::Factor) -> Bool

Whether two design rows differing in f's value must never share a physical plate.

is_registered_organism(name::Symbol; org_context=CHESSCore) -> Bool

Structural check: is name an organism CHESSCore already knows about (via CHESSCore.orgparse/org_context)? Mirrors is_registered_reagent – this is how an :organism-destination column is recognized, by the column name itself identifying the organism/strain, not by a registered Factor.

is_registered_reagent(name::Symbol; reagent_context=CHESSCore) -> Bool

Structural check: is name a reagent CHESSCore already knows about (via CHESSCore.reagentparse/reagent_context)? This is how a :reagent-destination column is recognized – by looking it up against whatever's already registered, not by registering a Factor per reagent (there could be hundreds).

layout(experiment::Experiment)

Convenience accessor for get_parameter(experiment, :layout).

parse_design(path::AbstractString; column_map=DesignColumnMap(), reagent_context=CHESSCore, org_context=CHESSCore) -> Experiment

Read a CSV design at path into an Experiment. Applies column_map's header translation (raw header -> canonical factor name) and value translation (raw value -> canonical value, for open-vocabulary categorical factors), then validates that every resulting column resolves to either a reagent CHESSCore already knows or a registered Factor – an unregistered column is a hard error here, a deliberate departure from ParameterKind's lenient "unregistered keys are just a raw lookup" stance, since a design has to be fully understood before it's usable.

column_map itself is stored as :column_map metadata on the returned Experiment, so a persisted design's raw-string interpretation stays auditable rather than known only at parse time.

partition_by_blocking(experiment::Experiment) -> (blocking_cols::Vector{Symbol}, groups)

Group experiment.design's row indices by equality across every column that's a registered Factor with is_blocking(f) == true – two rows land in the same group iff they agree on every blocking factor's value. If there are no blocking factors at all, every row lands in one group together (nothing to keep apart). Assumes experiment has already been through expand_control_templates – every row here is expected to have a concrete blocking value.

Returns blocking_cols (the blocking column names, in the fixed sorted order used to build each group's key) alongside groups, a vector of (key, rows) named tuples – key is a Tuple of that group's shared blocking-factor values (in blocking_cols order), rows its design-row indices. Groups are ordered by their first row index, for determinism.

A row's group membership comes from its :_block_key value when expand_control_templates set one (non-missing) – otherwise from plain equality across blocking_cols, unchanged. This lets an expanded control row join the block it's meant to serve even when its own column values (deliberately fixed, e.g. "no carbon source") disagree with that block's real values.

populate_well_conditions(experiment; reagent_context=CHESSCore, org_context=CHESSCore) -> Experiment

Populate :well_conditions from an already-scheduled experiment (one with :layout set, via schedule_layout or schedule_blocked_layout) – keyed by (labware, well), holding each placed row's :organism-destination values (reusing resolve_stock's organism resolution, without building a Stock) and every non-blocking :condition-destination value (a blocking :condition factor is guaranteed uniform per plate and belongs in :plate_conditions instead – see schedule_blocked_layout). Works on any scheduled Experiment regardless of how it was scheduled, since it only reads :layout and experiment.design, never RunMaps/ PlateMaps directly.

qc_method(name::Symbol) -> Type{<:QCMethod}

Look up a registered QCMethod subtype by name.

qc_methods() -> Vector{Symbol}

Names of every currently registered QCMethod.

register_factor!(f::Factor) -> Factor

Register f under f.name, making it resolvable via get_factor. Raises DuplicateRegistrationError if f.name is already registered.

register_parameter!(name::Symbol, type::Type; default=missing, validator=nothing) -> ParameterKind

Register a ParameterKind under name, making it resolvable via get_parameter. Raises DuplicateRegistrationError if name is already registered.

register_qc_method!(name::Symbol, ::Type{T}) where {T<:QCMethod}

Register a concrete QCMethod subtype under name, making it resolvable via qc_method. Raises DuplicateRegistrationError if name is already registered.

resolve_conditions(row, condition_names) -> Dict{Symbol,Any}

Resolve every :condition-destination factor in one design row into a plain Dict{Symbol,Any} – deliberately never a CHESSCore.Attribute (attributes are for things the physical object can actually be made to have; a plate can't be set to a temperature the way a well can be inoculated). Validates a CategoricalFactor's value against its registered levels.

resolve_stock(row, reagent_names, organism_names;
              units, reagent_context=CHESSCore, org_context=CHESSCore,
              total_volume=missing, solvent=nothing)
    -> (stock::CHESSCore.Stock, organisms::Dict{Symbol,Any})

Collectively resolve every :reagent- and :organism-destination factor in one design row into a single CHESSCore.Stock – this is deliberately just an interface to get every reagent/organism factor to a Stock, not an independent per-factor resolve step (matches how a Stock's composition is inherently a whole-row concept, and how vc/q-style tables are conventionally resolved as one unit in CHESSCore itself).

units is a Dict{Symbol,String} (one unit per reagent and organism factor – see DesignColumnMap).

total_volume/solvent implement the solvent shortcut: solvent (a reagent name, e.g. :water) fills whatever's left of total_volume after every other explicit liquid reagent factor already present is accounted for – so a design doesn't have to spell out its solvent as an explicit factor just to "bring to volume." If a reagent factor needs a total volume (a concentration or molar amount) and none is supplied, resolution errors rather than silently failing.

Organism factors work like reagent factors: oname (an entry of organism_names) IS the organism/strain identity (resolved via orgparse), and row[oname] is a CHESSCore.Biomass magnitude (paired with units[oname]) fed into CHESSCore's Biomass * Organism constructor – missing means no inoculum for that organism in this row.

Returns the built Stock (promoted to a Culture if any organism factor resolved) and a Dict{Symbol,Any} recording each organism factor's resolved biomass value per-well – this is what a technician-facing :well_conditions record is built from, independent of whether the same biomass value also reaches a scheduler via the returned Stock.

schedule_blocked_layout end

Partition experiment into blocking-homogeneous groups (see partition_by_blocking), schedule each group independently, and merge the results into one :layout – plus :plate_conditions and :well_conditions metadata. Only available once RunMaps and PlateMaps are also loaded (implemented in the CHESSExperimentsRunMapsExt package extension), same as schedule_layout.

schedule_layout(experiment, pa)

Schedule experiment's design-matrix rows onto a solved plate layout, returning a new Experiment with :layout metadata populated. Only available once RunMaps and PlateMaps are also loaded (implemented in the CHESSExperimentsRunMapsExt package extension) – neither RunMaps nor PlateMaps has any dependency on CHESSExperiments.

with_parameter(experiment::Experiment, name::Symbol, value) -> Experiment

Return a new Experiment with metadata[name] = value set (copy-on-write; experiment itself is unmodified). Does not require name to be pre-registered – writes stay as flexible as a raw Dict; only get_parameter reads enforce the registry.

CHESSExperimentsRunMapsExt

Schedules an Experiment's design-matrix rows onto one or more solved PlateMaps.PlateMaps, using a RunMaps.RunMap for the run/control relationship structure. Neither RunMaps nor PlateMaps depends on CHESSExperiments – this extension only loads when all three packages are loaded.

Node identity convention: the RunMap passed in must register its run nodes as exactly 1:nrow(experiment.design) (Int identities, one per design row). This is the whole trick that replaces PlateArrays' separate run_index field: a well's PlateMap occupant is its run_index when that occupant is one of these Int nodes – there is no ordering to compute or validate beyond "did every design row's node get placed exactly once."

The :layout metadata this produces keeps CHESSExperiments.LAYOUT_COLUMNS' exact shape (well,row,col,run,positive,negative,run_index,labware,location_id,metadata), derived from PlateMaps.DataFrame(::PlateMap, ::RunMap)'s occupant/incoming_roles columns, so every existing CHESSQC consumer (control_bitmatrix, data_correction, control_summary, ...) needs no changes.

schedule_blocked_layout(experiment::CHESSExperiments.Experiment, wells::BitMatrix, placeable_roles;
                         plate_solver="greedy", reagent_context=CHESSCore, org_context=CHESSCore,
                         kwargs...) -> CHESSExperiments.Experiment

Expand every control template (see CHESSExperiments.expand_control_templates), partition the result by every blocking Factor (see CHESSExperiments.partition_by_blocking), and schedule each blocking-homogeneous group independently – its own RunMap (built directly from the design's :control_role/:duplicates columns, see _build_partition_runmap) and its own PlateMaps.schedule_platemap call, which may itself split one group across multiple physical plates on capacity grounds. Since every node (including duplicate/control auxiliary nodes) is already identified by its real (expanded) design-row index – or a (row, copy) tuple for a duplicate well – no local-to-global renumbering is needed merging groups back together, unlike the old schedule_uniform_controls-based approach.

Returns a new Experiment (built from the expanded design, since control templates may have produced more rows than the original) with :layout, :plate_conditions (one entry per resulting labware, holding that group's shared blocking-factor values), :run_map/:plate_maps (one RunMap per partition, one labware => PlateMap pair per resulting plate – persisted rather than discarded, since CHESSProcessing.aggregate needs the same RunMap later to know what to average), and :well_conditions (via CHESSExperiments.populate_well_conditions) all set.

A blocking group that can't be scheduled (a control linked to samples spanning more physical wells than wells' capacity allows even after splitting across plates, say) surfaces as an ArgumentError naming which group failed, wrapping whatever RunMaps/PlateMaps raised – no new feasibility pre-check is done; those functions already refuse to silently misplace anything.

schedule_layout(experiment::CHESSExperiments.Experiment,
                 plates::AbstractVector{<:Pair{<:Union{AbstractString,Missing},<:PlateMaps.PlateMap}},
                 rm::RunMaps.RunMap) -> CHESSExperiments.Experiment

Concatenate each labware_name => pm pair's layout (tagged with labware = labware_name) into a single layout, and return a new Experiment with :layout metadata populated. rm is the RunMap every pm in plates was scheduled from (shared across plates for a multi-plate schedule). Every design row's node (1:nrow(experiment.design)) must appear across the given plates exactly once, with none missing, duplicated, or unplaced.

schedule_layout(experiment::CHESSExperiments.Experiment, pm::PlateMaps.PlateMap, rm::RunMaps.RunMap;
                 labware=missing) -> CHESSExperiments.Experiment

Single-plate convenience form: schedules experiment onto the one plate pm (already produced by PlateMaps.schedule_platemap from rm), tagged with labware.

schedule_layout(experiment::CHESSExperiments.Experiment, rm::RunMaps.RunMap, wells::BitMatrix,
                 placeable_roles; labware_names=nothing, kwargs...) -> CHESSExperiments.Experiment

Convenience form that schedules rm itself via PlateMaps.schedule_platemap (auto-splitting across as many wells-shaped plates as needed), then builds the layout. labware_names, if given, must have one name per plate schedule_platemap returns (in order); nothing leaves every plate's labware as missing (fine for a single, unnamed plate).

_build_partition_runmap(design, rows) -> RunMaps.RunMap

Build one blocking-partition's RunMap directly from the (already-expanded) design, replacing the old RunMaps.schedule_uniform_controls-based synthesis. Every row in rows (sample or control alike) gets its own node, using its plain design-row index – unchanged from the original node-identity convention. A row with _duplicate_count > 1 gets n-1 extra auxiliary nodes (a (row, copy) named tuple – can never collide with a real Int row index), each linked to the row's own node via a :duplicate-typed edge; CHESSProcessing.aggregate already knows how to collapse these (RunMaps.linked_runs(run_map, anchor; type=:duplicate)), no new grouping mechanism needed.

Every sample row gets linked to every one of a control row's replicate nodes – the control's own anchor and every one of its duplicate nodes, not just the anchor – labeled with the control's own role. Both parts of this matter:

• Direction: RunMaps.linked_runs(map, anchor; type) returns nodes anchor points to (RunMaps/src/query.jl's own docstring: "run is the subject of the edge"), and CHESSProcessing.aggregate/normalize/flag all query with the sample as anchor – so the edge must run sample -> control, not control -> sample, for a sample's control group to be discoverable at all.
• Full replicate set, not just the anchor: if a sample only saw the control's anchor node, its :positive/:negative-linked "group" would have exactly one member – normalize would then use only one of the control's 12 replicate wells (not their mean) as the control value, and flag's cv_threshold rule (which needs >=2 values to compute a CV at all) would see nothing to judge. Every sample must see the whole replicate set directly for both to work as intended.

This is the same dense every-run-to-every-control linking schedule_uniform_controls already did, just sourced from real, user-designated rows, covering their full duplicate set, and pointed the direction CHESSProcessing expects.

_duplicate_count(design, i) -> Int

Total physical wells design row i should get, including its own anchor well – missing/1 (or :duplicates absent entirely) means just the one well.