CHESSDatabase API Reference

location_reconstruction_index

Default (empty) value for reconstruct_contents's loc_index accumulator: a map from LocationID to every (SequenceID, Location) snapshot recorded for it during a single reconstruction pass.

This used to be a single DataFrame scanned in full (filter + sort) on every lookup via find_most_recent_location – correct, but with a reconstruction touching N locations across the replay, that meant up to 2N linear scans of a structure that itself grows to ~2N rows, i.e. O(N²) total work. A well fed by a heavily-reused hub location (thousands of transfers touch that hub over a lab's lifetime) turned this into the dominant cost of reconstruction – see the reconstruction scaling benchmark's REPORT.md for the measurement (99.8% of wall time in this bookkeeping, only 0.2% in the SQL query that finds the transfers in the first place). Keying by LocationID up front makes each lookup scan only that one location's own (typically small) history instead of everything ever touched in the whole reconstruction.

Invariant: each location's entry vector is always appended in non-decreasing SequenceID order. Every push_location! call site in reconstruct_contents.jl is either a pre-replay-loop bootstrap entry (seq <= foot) or a replay-loop entry (seq == row.SequenceID, with transfers fetched at SequenceID >= foot+1 and iterated in ascending SequenceID order). find_most_recent_location relies on this invariant to binary-search each location's vector instead of scanning it – see its docstring. Any future push_location! call site must preserve non-decreasing order per location, or that binary search silently returns wrong results instead of erroring.

mutable struct Barcode

A physical barcode string, with an optional name and the location_id of the location it is attached to (missing until assigned). See assign_barcode!.

struct Run
Run(loc::Location, exp_id::Integer, controls=Integer[], blanks=Integer[])

One run of an experiment: the location it was performed in, the experiment's ID, and the location IDs of its controls and blanks. Record it with upload_run.

_apply_content_observations!(all_locs, rows)

Replay one location's observations at one sequence point (rows share a LocationID and SequenceID) on top of its most recent reconstructed state, and record the result at that sequence point. Skipped when that location's state is only known from a point at or after the observation – a cache written there already reflects it.

_reconstruction_transfer(donor,recipient,quantity,configuration::String="")

Internal, non-exported helper used only by the ledger-replay simulation below. Returns new (donor,recipient) objects reflecting a transfer of quantity, without mutating the originals — the replay needs to keep both the pre- and post-transfer versions of each well in its own history log (all_locs), so it can't just mutate in place like transfer! does.

This is deepcopy-based like the old public non-mutating transfer (removed — see reconstruct_location/build_location for the actual preview pattern), but it's safe here specifically because src/dest in this context are always standalone reconstruction shells (parent === nothing, no children) built directly by t(loc_id,n) a few lines up — never part of a live connected tree — so deepcopy has nothing extra to copy.

append_ledger()

Add an entry to the end of ledger and return the ID of that entry.

assign_barcode!(barcode::Barcode, location::Location; instrument=nothing)

Attach barcode to location in memory. Throws an error if the barcode is already attached to a different location. update_barcode records the assignment in the database.

assign_barcode(barcode::Barcode, location::Location) -> Barcode

Non-mutating assign_barcode!: return a copy of barcode attached to location.

backfill_observations()

One-off migration for a database written before observations existed, where some state is known only from a cache (for example a bottle's contents, cached when it was committed). Returns the number of ledger entries added.

Walks every cache in sequence order and compares it with what the ledger alone predicts at that point. Wherever they differ, the cached state is recorded as observations on a new ledger entry inserted right after the cache's, and the caches from there on are repaired. Afterwards every cache can be recomputed from the ledger.

Labware membership is structural rather than recorded, so it stays with the caches: which labware a well (or anything else filling a labware slot) belongs to, and the grid of slots inside a labware.

barcode(x::Barcode) -> String

The barcode string itself.

cache(loc::Location, sequence_id=nothing, time=Dates.now())

Store a snapshot of loc's current parent, children, environment, lock and activity state, and (for a well) contents, stamped with the ledger entry at sequence_id (default: the end of the ledger). Later reconstructions start from the snapshot instead of replaying the whole history. loc and everything inside it must already be committed. Caching is always explicit; nothing caches automatically. See the Caching & Repair manual chapter.

cache_repair(ledger_id::Integer)

Repairs caches affected by updates, insertions, and deletions from the ledger.

When a cache is invalidated by an operation, a new cache is written to replace the old cache in future reconstructions.

The old cache will still be utilized for reconstrutions triggered at a time before the cache was repaired.

commit_location!(loc::Location) -> Location

Commit an uncommitted (!is_committed(loc)) location subtree to the database, preserving its current structure, attributes, lock/active state, and (for Wells) stock contents.

Returns a new, independent, committed Location – loc itself is left unchanged and still uncommitted (location_id/name/kind are immutable fields on every concrete Location subtype, so committing in place is not possible). Use the returned value going forward, the same way reconstruct_location returns an independent object rather than mutating its input.

Already-committed locations embedded in an otherwise-uncommitted subtree are left as-is (reattached, not re-committed) – commit_location! is safe to call on a mixed tree.

Whatever the new locations hold beyond a fresh location's defaults (stock, cost, attributes, lock and activity state, and which location each child sits in) is recorded as observations on a single new ledger entry, so the ledger alone can reconstruct it. The new locations are then cached at that entry.

See also: build_location, release_location, UncommittedLocationError, observe.

commit_run_location!(design_id::Integer, labware::AbstractString, well::AbstractString, location_id::Integer)

Fuse a scheduled well with a real, committed CHESSCore.Location by updating its RunAssignments row's LocationID – the opportunistic-fusion update: nothing else about the row changes. (labware, well) together identify the row – well names (e.g. "A1") repeat across plates once a design spans more than one, so labware is required to disambiguate.

connect_SQLite(path)

Open the SQLite database at path and make it the database that every CHESSDatabase function reads and writes. SQLite creates the file if it does not exist; use create_db to set up the CHESS tables in a new file. Calling it again switches to a different database.

A database created before the Chemicals and CompositionRules tables became the Species and Dissolutions tables is still readable only after migrate_chemistry! is run on it; connect_SQLite warns when it finds one.

create_db(path)

Create a new CHESS database at path: every table described on the Database Architecture manual page, plus the first ledger entry (sequence ID 1). The file must not already contain these tables. create_db does not connect to the new database; call connect_SQLite afterward.

db_time(time::DateTime) -> Float64

Convert time to the Unix-time number the database stores. See julia_time.

encumber(protocol_id::Integer, fun::Function, args...) -> Integer

Reserve a planned operation under protocol protocol_id without writing it to the ledger. fun is applied to args in memory immediately, and the reservation is recorded in the matching Encumbered* table. Returns the new encumbrance's ID. Nothing marks the encumbrance complete when the real operation later happens; see upload_encumbrance_completion.

encumber_cache(encumberid::Integer, loc::Location)

Store a snapshot of loc's state under encumbrance encumberid, the encumbrance counterpart of cache, for reconstructions that include planned operations.

execute_db(query::String)
execute_db(query::String, params)

Run a SQL statement that changes the connected database (INSERT, UPDATE, CREATE, ...). params fills the statement's ? placeholders; prefer it over building SQL with string interpolation. Foreign-key constraints are switched on for every call. Throws an error if no database is connected (see connect_SQLite).

See also: query_db for statements that return rows.

fetch_content_cache(location_id::Integer,starting::Integer,ending::Integer,time::DateTime=Dates.now();encumbrances=false)

Look up location_id's most recent cached Stock within starting..ending. Returns (stock, cost, foot).

stock is nothing when no CachedContents row exists in that window at all – distinct from a row that exists and legitimately caches an empty stock (which returns Empty(), not nothing). Callers (see reconstruct_contents) rely on this distinction: nothing means "we don't actually know this location's content here, go reconstruct it," while Empty() means "confirmed empty as of this cache point."

find_most_recent_entry(index, location_id, sequence_id)

Like find_most_recent_location, but return the (SequenceID, Location) entry itself, so a caller can tell what sequence point the found state reflects.

find_most_recent_location(index, location_id, sequence_id)

Return the most recently pushed Location for location_id with SequenceID <= sequence_id, or nothing if none qualifies. Relies on the same sortedness invariant as the 2-arg method: since entries are always appended in non-decreasing SequenceID order, a binary search (searchsortedlast) finds the rightmost qualifying entry in O(log k) instead of a linear scan. Ties (more than one entry at the same sequence_id) resolve to the last one pushed, matching the insertion-order tie-breaking a linear max-scan would produce.

find_most_recent_location(index, location_id)

Return the most recently pushed Location for location_id in a loc_index accumulator (see location_reconstruction_index), or nothing if it has no entries. Relies on that type's documented sortedness invariant: since entries are always appended in non-decreasing SequenceID order, the most recent one is simply the last one – O(1), no scan needed.

generate_location(kind::LocationKind,name::String=string(UUIDs.uuid4()),child_namer::Vararg{Function}=plate_namer)

Generate a real, database-tracked kind Location and fill it with empty wells if applicable.

Optionally add a variable number of functions to recursively name children of generated labware. naming functions should take two arguments row and col and return a string. The default is plate_namer

See also: plate_namer, build_location.

get_all_attributes() -> DataFrame

Every attribute kind recorded in the connected database's Attributes table, with its base unit.

get_all_runs(exp_id::Integer) -> Vector{Run}

Every run recorded for experiment exp_id.

get_barcode(barcode::String) -> Barcode

Look up a barcode in the connected database. Throws an error if it has not been recorded.

get_component_id(str::Organism)

Return the Components.ID/Organisms.ComponentID for str, uploading it first (via upload_component) if no row with a matching natural key (genus/species/strain) already exists.

get_component_id(reagent::Reagent)

Return the Components.ID/Reagents.ComponentID for reagent, uploading it first (via upload_component) if no row with a matching natural key (name/type/molecular weight/ density/pubchem ID) already exists. Reagent has a small, fixed set of scalar fields, so identity is looked up directly by those fields rather than via a content hash.

get_content_observations(locs::Vector{<:Integer},starting::Integer=0,ending::Integer=get_last_sequence_id(),time::DateTime=Dates.now())

Every component and cost observation on locs between starting and ending sequence points, ordered by SequenceID. Component rows have a missing Cost, and cost rows a missing ComponentID/Quantity/Unit.

get_design(design_id::Integer) -> CHESSExperiments.Experiment

Reconstruct the Experiment persisted under design_id, including its :layout metadata (built from RunAssignments) if it was scheduled.

get_instrument_settings(instrument_id::Integer,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now())

Resolve instrument_id's current value per Setting name as of sequence_id/time, as a DataFrame (columns Setting, Value, SequenceID). Unlike get_reads, this does collapse to latest-wins per Setting – instrument settings behave like Attribute (a single current value), not like Read (many coexisting values) – mirroring get_attribute_caches's ledger_subset idiom.

get_last_encumbrance_id(protocol_id::Integer) -> Integer

The ID of the most recently created encumbrance in protocol protocol_id.

get_last_ledger_id(time::DateTime=Dates.now())

Return the most recent ledger id entry at or before a certain time

get_last_protocol_id(experiment_id::Integer) -> Integer

The ID of the most recently created protocol in experiment experiment_id.

get_last_sequence_id(time::DateTime=Dates.now()) -> Integer

The newest sequence ID in the ledger as of time: the current end of the history.

get_location_info(id::Integer) -> (name, constructor)

Look up location id in the database. Returns its name and a function constructor(id, name) that builds a bare location of the right kind (no parent, children, or contents yet). Throws an error if id is not in the database.

get_reads(location_id::Integer,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now())

Return every Reads row for location_id up to sequence_id/time, as a DataFrame. Unlike get_*_caches (e.g. get_attribute_caches), there is no latest-wins collapsing here – reads never supersede each other, only accumulate (see Read's docstring).

get_run(run_id::Integer) -> Run

Look up a run in the connected database. Throws an error if it is not there.

get_sequence_id(ledger_id::Integer) -> Integer

The sequence ID (position in the history) of ledger row ledger_id. Several ledger rows share a sequence ID when an entry has been revised with replace_ledger.

get_species(id::Integer)

Reconstruct a bare Species from the Species table by ID – the direct, standalone counterpart of get_component for Reagent/Organism. Species reconstruction otherwise only happens inline inside restore_dissolution! as part of rebuilding a Reagent's Dissolution; this gives it the same testable, direct path.

get_species_id(spc::Species)

Return the Species.ID for spc, uploading it first (via upload_species) if no row with a matching natural key (name/charge/formula/stored molecular weight) already exists.

get_transfer_ancestors(locs::Vector{<:Integer},starting::Integer=0,ending::Integer=get_last_sequence_id(),time::DateTime=Dates.now();encumbrances=false)

Find every transfer relevant to reconstructing locs' contents between starting and ending sequence points.

The result has a Core column (1/0). Core=1 rows are the genuine ancestor closure: locs themselves plus every location transitively feeding them (found the traditional way, recursively following Source = <already-found Source>.Destination). Core=0 ("collateral") rows are other outgoing transfers of a Core=1 location, to a destination that never itself feeds back into the closure – needed so replay can correctly account for stock a core location sent away to something reconstruct_contents doesn't otherwise care about (see the module docs above x's definition for why this ever mattered). Collateral destinations never need their own correct reconstruction – reconstruct_contents bootstraps them as bare Empty() stubs instead of recursing, which is what keeps a busy/shared well's reconstruction from pulling in a large, mostly-irrelevant slice of the whole ledger's history: the query still has to look at every collateral edge (that's inherent to correctly zeroing out a core well's balance), but no collateral destination ever triggers its own expensive reconstruction.

hash_bytes(x)

Return hash(x) (a UInt64) as its raw 8 bytes (Vector{UInt8}), for binding as a BLOB parameter in hash-dedup columns (CachedStocks/CachedChildSets/CachedAttributeSets). Binding this directly (rather than interpolating hash(x) as a bare numeric literal) stores and compares the exact 64-bit value regardless of magnitude – interpolating a UInt64 above typemax(Int64) as a literal silently loses precision (SQLite coerces it to REAL).

insert_ledger()

Increase the SequenceID value for all ledger entries whose SequenceID Value is greater than or equal to sequence_id, then insert a ledger new entry into the sequence at sequence_id

isa_observation(ledger_id::Integer)

true if the ledger entry ledger_id holds at least one observation.

julia_time(time::Real) -> DateTime

Convert a Unix-time number stored in the database back to a DateTime. Inverse of db_time.

migrate_chemistry!()

Convert the connected database from the old Chemicals and CompositionRules tables to the Species and Dissolutions tables: rename Chemicals to Species and add its nullable Formula column, rename CompositionRules to Dissolutions, and rename the ChemicalID column of Dissolutions to SpeciesID. Existing rows keep their stored molecular weights and have no formula. Returns true if the database was converted and false if there was nothing to do (already migrated, or a new database). Run it once per existing database; the changes are made in a single transaction.

observation_discrepancies(location_ids; sequence_id=get_last_sequence_id(), time=Dates.now())

Compare every observation of location_ids up to sequence_id with what history predicted just before it, and return the ones that differ as a DataFrame with columns LedgerID, SequenceID, LocationID, Facet ("component", "cost", "attribute", "parent", "locked" or "active"), Key (the component or attribute, else missing), Predicted and Observed.

Attributes are compared with the location's effective environment, so a value it inherits counts. Parents are reported as location ids (missing for none).

observe(loc::Location, facet...; read_ledger_id=nothing, ledger_id=nothing, time=Dates.now(), instrument=nothing, instrument_time=nothing)
observe(loc::Location; recursive=true, read_ledger_id=nothing, ledger_id=nothing, time=Dates.now(), instrument=nothing, instrument_time=nothing)

Record an observation of loc in the database: a statement of what its state is at a ledger position, whatever history predicted there. Later operations replay on top of it, and nothing before it can change the observed value. Returns the ledger id of the observation.

With facet arguments, observe one fact, e.g. observe(well, rgt"glucose", 2u"g") (see observe! for every form); loc is updated in memory too.

With no facet, observe all of loc as it currently is in memory: every stock component (and a zero for any component history predicts but loc lacks), its cost, its own attributes (and missing for any own attribute history predicts but loc lacks), its parent, lock and activity. recursive also observes every location inside loc. All of it shares one ledger entry.

By default the observation is appended to the ledger. Pass the ledger_id of an entry made with insert_ledger to record it at an earlier point: the change is then validated and the caches after it repaired, as with update.

See also: observation_discrepancies, backfill_observations.

process_update(ledger_id::Integer)

The checks update runs after recording an amendment at ledger_id: that a replacement is the same kind of operation as the entry it replaces, that the amended history can still be replayed (validate), and repair of any caches the amendment made stale (cache_repair).

push_location!(index, location_id, sequence_id, loc)

Record a (sequence_id, loc) snapshot for location_id in a loc_index accumulator (see location_reconstruction_index). The push!(all_locs, (id, seq, loc)) counterpart from the old DataFrame-based accumulator.

query_db(query::String) -> DataFrame
query_db(query::String, params) -> DataFrame

Run a SQL query against the connected database and return the result as a DataFrame. params fills the query's ? placeholders. Foreign-key constraints are switched on for every call.

See also: execute_db.

query_join_vector(entry::Vector{String})

Return ("(?,?,...)", Tuple(entry)) – a parameterized IN (...) clause fragment (one ? per element) alongside the params to pass to execute_db/query_db, instead of joining raw strings directly into SQL text.

reconstruct_activity(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_activity(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's active state for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_activity!.

reconstruct_activity!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_activity!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the active state of existing locations to their reconstructed state. Arguments work as in reconstruct_activity.

reconstruct_attributes(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_attributes(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's own attributes for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_attributes!.

reconstruct_attributes!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_attributes!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the own attributes of existing locations to their reconstructed state. Arguments work as in reconstruct_attributes.

reconstruct_children(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_children(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's children (as bare locations, not themselves reconstructed) for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_children!.

reconstruct_children!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_children!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the children of existing locations to their reconstructed state. Arguments work as in reconstruct_children.

reconstruct_contents(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_contents(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild the stock and cost of wells from the ledger's transfer history, returning new locations for the given IDs. reconstruct_contents! writes the result into existing Locations instead. IDs that are not Wells come back as bare locations with no contents.

A well's contents can depend on every well that ever transferred into it, so reconstruction starts from the most recent cached stock of each well and replays only the transfers since then. Passing several wells at once shares that work.

The optional arguments work as in reconstruct_location!: sequence_id and time choose the point in history, max_cache limits which caches may be used, and encumbrances=true includes planned future operations.

reconstruct_contents!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_contents!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the stock and cost of each Well in locations to its reconstructed state. Other locations are left alone. Arguments work as in reconstruct_contents.

reconstruct_environment(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_environment(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's chain of ancestors and their attributes, so that environment gives the inherited environment for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_environment!.

reconstruct_environment!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_environment!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the parent chain and attributes of existing locations to their reconstructed state. Arguments work as in reconstruct_environment.

reconstruct_location(location_id::Integer,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now(),max_cache::Integer=sequence_id;encumbrances=false)

Reconstruct the entire state of location in a CHESS Database using only an integer location_id

Optional Arguments

1. sequence_id: set the reconstruction to happen at a particular sequence point in the ledger. Default is the current maximum sequence ID in the Database.
2. time: only consider operations that were recorded in the CHESS ledger before time. time uses Dates.jl to standardize the time formatting. Default is Dates.now()
3. max_cache: only consider caches up to a certain sequence point and force the algorithm to simulate thereafter. Default is the current value of sequence_id. This feature is used for cache repair.

Keyword Arguments

1. encumbrances=false: Include encumbrances in the reconstruction. Encumbrances are future operations that are not in the ledger but have been planned.

Note: Combining these optional arguments allows users to traverse the state of any location at any time, past or present. Encumbrances are enforced as they would have been at the sequence_id and time of the reconstruction.

Previewing hypothetical changes: reconstruct_location builds a fresh, independent object graph from the ledger on every call — mutating the result (e.g. with move_into!, drain!, lock!, ...) never touches any other object, live or otherwise, since nothing else shares its identity. This makes it the way to preview a hypothetical change to a real, already-persisted location: reconstruct a disposable copy, mutate it however you like, inspect it, then just let it go — there's nothing to "restore" since the live object you started from was never touched. Call reconstruct_location again for another independent snapshot. For hypothetical locations that don't exist yet, see build_location instead.

See Also: reconstruct_location!, build_location

reconstruct_location!(loc::Location,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now(),max_cache::Integer=sequence_id;encumbrances=false)

Reconstruct the entire state of a Location location in a CHESS Database.

Optional Arguments

1. sequence_id: set the reconstruction to happen at a particular sequence point in the ledger. Default is the current maximum sequence ID in the Database.
2. time: only consider operations that were recorded in the CHESS ledger before time. time uses Dates.jl to standardize the time formatting. Default is Dates.now()
3. max_cache: only consider caches up to a certain sequence point and force the algorithm to simulate thereafter. Default is the current value of sequence_id. This feature is used for cache repair.

Keyword Arguments

1. encumbrances=false: Include encumbrances in the reconstruction. Encumbrances are future operations that are not in the ledger but have been planned.

Note: Combining these optional arguments allows users to traverse the state of any location at any time, past or present. Encumbrances are enforced as they would have been at the sequence_id and time of the reconstruction.

See Also: reconstruct_location

reconstruct_lock(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_lock(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's locked state for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_lock!.

reconstruct_lock!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_lock!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the locked state of existing locations to their reconstructed state. Arguments work as in reconstruct_lock.

reconstruct_parent(location_id::Integer, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Location
reconstruct_parent(location_ids::Vector{<:Integer}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false) -> Vector{<:Location}

Rebuild each location's parent for the given location IDs, as of sequence_id and time, and return new locations holding that state. reconstruct_location rebuilds everything at once; the arguments work the same way (see reconstruct_location!).

See also: reconstruct_parent!.

reconstruct_parent!(location::Location, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)
reconstruct_parent!(locations::Vector{<:Location}, sequence_id=get_last_sequence_id(), time=Dates.now(), max_cache=sequence_id; encumbrances=false)

Set the parent of existing locations to their reconstructed state. Arguments work as in reconstruct_parent.

reconstruct_reads(location_id::Integer,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now(),max_cache::Integer=sequence_id;encumbrances=false)

Reconstruct a location by location_id and populate its reads. See reconstruct_reads!.

reconstruct_reads!(loc::Location,sequence_id::Integer=get_last_sequence_id(),time::DateTime=Dates.now(),max_cache::Integer=sequence_id;encumbrances=false)

Populate loc's reads collection from the database, up to sequence_id/time, sorted by InstrumentTime (falling back to the upload Time when InstrumentTime is unavailable) – the sort is what makes reads(loc) interpretable as a time series. max_cache/encumbrances are accepted (and ignored) purely so reconstruct_location! can call every reconstruct_*! uniformly – reads have no cache/encumbrance layer (see get_reads).

release_location(loc::Location) -> Location

Build an uncommitted copy of loc's subtree – same name/kind/structure/attributes/lock/active state (and, for Wells, stock), but with location_id === nothing throughout. The inverse of commit_location!. Never touches the database – operates purely on loc's already- materialized in-memory state (e.g. from reconstruct_location).

Typical use: merging structures from separate databases (only one connect_SQLite connection can be live at a time) – reconstruct the subtree(s) you want from the source database, release_location each one to strip the source IDs, recombine freely (e.g. build_location a new root and move_into! each released piece into it), reconnect to the target database, and commit_location! the merged result.

See also: commit_location!, reconstruct_location, build_location.

replace_ledger(sequence_id::Integer)

Create a new revision of an already-used SequenceID slot – the ledger's "amend" operation (see the module-level note above append_ledger). Asserts sequence_id is currently occupied; use append_ledger or insert_ledger for a slot that doesn't exist yet.

restore_dissolution!(reagent::Reagent, component_id::Integer)

Restore reagent's dissociation rule from the Dissolutions/Species tables (see upload_dissolution, src/database/uploads.jl), registering it via set_dissolution! so a reagent reconstructed from the database behaves identically to one defined by its original lab module in the current session – no longer dependent on that module being loaded. No-op if nothing was persisted for component_id (e.g. locations uploaded before this table existed).

sql_commit(name::String)

Commit the named SQL savepoint on the connected database. Most code uses sql_transaction instead.

sql_rollback(name::String)

Roll back to the named SQL savepoint on the connected database. Most code uses sql_transaction instead.

sql_transaction(f::Function)

Run f() inside a SQL transaction on the connected database: if f throws, every write it made is rolled back. Returns f()'s value. upload and update use this so an operation and its database record succeed or fail together.

sql_where(pairs::Pair{String}...)

Build a parameterized WHERE-clause fragment from (column, value) pairs, returning (sql_fragment, params) – pass sql_fragment in the query text and params as execute_db/query_db's second argument. missing compiles to "col IS NULL" with no bound parameter (= ? never matches NULL under SQL's three-valued logic, whether the NULL comes from a literal or a bound value); every other value compiles to "col = ?" with the value collected into params.

update(fun::Function, args...; replace=nothing, insert=nothing, time=Dates.now(), instrument=nothing, instrument_time=nothing) -> Integer

Run an operation and record it at a chosen point in the ledger, then validate the result and repair any caches it invalidates. Returns the ledger ID the operation was written to.

update works like upload, but is meant for amending history:

• replace=s records a new revision of the entry at sequence ID s (see replace_ledger). The replacement must be the same kind of operation as the entry it replaces, otherwise an error is thrown.
• insert=s adds a new entry at sequence ID s, moving later entries back by one (see insert_ledger).
• With neither, the operation is appended at the end of the ledger.

The ledger row, the operation, validation, and cache repair all happen in one SQL transaction, so if any step fails the database is left exactly as it was. fun is also applied to the in-memory objects in args, as with upload, and that change is not undone on failure; reconstruct from the database to see the recorded state.

ledger_id= is still accepted in place of replace/insert, for a ledger row allocated beforehand, but a row allocated that way is not rolled back if update fails.

See also: replace_ledger, insert_ledger.

Examples

update(transfer!, well_a, well_b, 1u"g"; replace=54)

update_barcode(bc::Barcode, loc::Location; kwargs...)

Record in the database that bc is attached to loc. This is the database write for assign_barcode! (see upload_operation).

update_ledger(sequenceID::Integer)

Low-level primitive: insert a new Ledger row at sequenceID, stamped with the current time, and return its ID. Both append_ledger and insert_ledger funnel through this. Called directly on an already-occupied sequenceID, it's the ledger's "replace" operation – prefer replace_ledger for that, since it asserts the slot already exists instead of silently succeeding either way.

upload(fun::Function,args...;ledger_id=nothing,time=Dates.now(),instrument=nothing,instrument_time=nothing)

Execute and upload a CHESS operation to a CHESS Database. If an error occurs in either execution or uploading, upload will return an error and rollback any changes to the database

See upload_operation for the list of supported operations. If instrument is given, it's passed to fun itself (which gates capability, e.g. via CHESSCore._check_capability, for the four operations that support it) and used to compute instrument_id for attribution when persisting – upload/update don't re-check capability themselves, since it already happened inside fun.

#Examples

julia> connect_SQLite("test_db.db")

julia> upload(transfer!,locA,locB,5u"g")
julia> upload(set_attribute!,locA,Temperature(10u"°C"))

upload_activity(location::Location)

Add an entry to the Activity table a Location

Locations can be toggled between an active and inactive state. Activity can be used to show or hide locations in user interfaces.

upload_barcode(bc::Barcode)

Record a barcode in the database, with its current location if it has one. A barcode already recorded is left unchanged.

upload_design(experiment::CHESSExperiments.Experiment, exp_id::Integer) -> Integer

Persist experiment (design matrix, layout if scheduled, metadata) under the given Experiments row and return the new Designs.ID. If CHESSExperiments.layout(experiment) is set, one RunAssignments row is uploaded per well.

upload_dissolution(reagent_component_id::Integer, reagent::Reagent)

Persist reagent's currently-resolved dissolution (dissolution(reagent).products) as Dissolutions rows – the DB counterpart of the in-memory dissolutions registry. Called whenever a Reagent component is uploaded, so a reagent reconstructed from the database (see get_component, reconstruct_contents.jl) can restore its dissociation rule without depending on the defining lab module being loaded in that session.

upload_encumbrance(protocol_id::Integer) -> Integer

Create an empty encumbrance in protocol protocol_id and return its ID. encumber calls this and then records the planned operation against it.

upload_encumbrance_completion(encumbrance_id::Integer, ledger_id::Integer)

Mark encumbrance encumbrance_id as completed by the operation recorded at ledger_id. This is always an explicit call: performing the real operation does not complete an encumbrance by itself.

upload_experiment(name, user::String, is_public=false; time=Dates.now()) -> Integer

Create an experiment record and return its ID. Protocols and runs belong to an experiment.

upload_instrument_setting(instrument::Location,setting::String,value; ledger_id, time, instrument_time)

Append a new revision of instrument's setting to InstrumentSettings – the ledger's "amend" operation for instrument settings, mirroring upload_environment_attribute's shape. value is stored as text (see the design note on InstrumentSettings – settings may be non-numeric). Returns ledger_id – since SequenceID can shift after the fact (insert_ledger/replace_ledger), resolve this revision's current sequence position later via get_sequence_id(ledger_id) rather than caching a SequenceID directly.

upload_observation(well::Well, component::StockComponent, quant; ledger_id, time, instrument_id, instrument_time, read_ledger_id)
upload_observation(loc::Location, attribute::Attribute; ...)
upload_observation(child::Location, parent::Union{Location,Nothing}; ...)
upload_observation(loc::Location, facet::Symbol, value; ...)

Persist one observation (see observe! for the meaning of each form). Pure persistence, like upload_transfer: the in-memory change is the caller's responsibility, normally through upload(observe!, ...), update(observe!, ...) or observe. read_ledger_id optionally links the observation to the ledger entry of the Read that produced it.

An attribute observation whose value matches what loc already inherits from its parent (and loc has no value of its own) is stored as "confirmed inherited": replay skips it, so loc keeps following its parent. The attribute is removed from loc's own attributes in memory to match.

upload_operation(fun::Function) -> Function

The database-writing function that upload and update call to persist an operation fun. Supported operations:

Throws a KeyError for any other function.

upload_protocol(exp_id::Integer, name, ledger_id_entered_at=get_last_ledger_id(), estimate=Dates.Time(0); enforce=true) -> Integer

Create a protocol named name in experiment exp_id: a group of encumbrances. estimate is the protocol's expected duration. With enforce=true, the protocol's encumbrances are included in reconstructions that ask for encumbrances. Returns the new protocol's ID. (exp_id, name) must be unique.

upload_read(loc::Location,read::Read; ledger_id, time, instrument_id, instrument_time)

Persist read (a Read) for loc. Pure persistence – matches upload_movement/upload_transfer/upload_environment_attribute: the in-memory mutation (record_read!) is the caller's responsibility (normally upload(record_read!,loc,read;instrument=...), which calls both). instrument_time is accepted and ignored – read's own read_time is what's stored as Reads.InstrumentTime, since a Read carries its own recorded time intrinsically. Value/Unit come from quantity(read): quantitative reads store a numeric string + unit; qualitative reads store the raw string with a NULL unit.

upload_run(run::Run) -> Integer

Record run in the database and return its ID.

upload_species(spc::Species)

Insert spc into the Species registry (the DB counterpart of Species dissociation identities – distinct from Reagents/Components, since a Species never participates in a Stock directly). Returns its Species.ID.

upload_transfer(source::Well,destination::Well,quant::Union{Unitful.Mass,Unitful.Volume},configuration::AbstractString="";ledger_id,time,instrument_id,instrument_time)

Commit a transfer of quant from source to destination.