Ciaren

SQL input — sqlInput

Read rows live from a database at run time, via a reusable Connection. Because the read happens on every run, scheduled flows always process fresh data.

  1. 1Input
    SQL Input
    orders table — live DB
  2. 2Clean
    Filter Rows
    status = shipped
  3. 3Transform
    Group By
    revenue by region
  4. 4Output
    File Output
    daily report

Use cases

  • Run a flow against the current contents of a production table on a schedule.
  • Pull a query result (a join or filter computed in the database) into a flow.
  • Read from MongoDB by selecting a collection.

Configuration

Config keyTypeRequiredDescription
connection_idstringYesThe connection to read from
modestringNotable (default) or query
tablestringConditionalTable name (required in table mode)
schemastringNoSchema the table lives in
querystringConditionalCustom SQL (required in query mode)

Generated Python code

python
import os
from sqlalchemy import create_engine

_engine_1 = create_engine(f"postgresql+psycopg://reader:{os.environ['PG_PASSWORD']}@host:5432/shop")
df_orders = pd.read_sql_table('orders', _engine_1)

Each run also snapshots the input to parquet for reproducibility. MongoDB sources use collection selection (no custom query).

Tips & common mistakes

  • Passwords are never embedded. Exported code resolves the secret at runtime from its reference scheme — os.environ[...] for a bare name or env:NAME, keyring.get_password(...) for keyring:NAME (recommended on desktop), or a file:/path read for a mounted secret file. See Connections for the full scheme list.
  • Test the connection first. Use the connection's Test action to confirm credentials and reachability before wiring it into a flow.

See also