Ciaren

SQL input

SQL input โ€” sqlInput

Read rows live from a database at run time, via a reusable Connection. Because the read happens on every run, scheduled flows always process fresh data.

SQL Input
orders table โ€” live DB
input
Filter Rows
status = shipped
clean
Group By
revenue by region
transform
File Output
daily report
output

Use cases

  • Run a flow against the current contents of a production table on a schedule.
  • Pull a query result (a join or filter computed in the database) into a flow.
  • Read from MongoDB by selecting a collection.

Configuration

Config keyTypeRequiredDescription
connection_idstringYesThe connection to read from
modestringNotable (default) or query
tablestringConditionalTable name (required in table mode)
schemastringNoSchema the table lives in
querystringConditionalCustom SQL (required in query mode)

Generated Python code

import os
from sqlalchemy import create_engine

_engine_1 = create_engine(f"postgresql+psycopg://reader:{os.environ['PG_PASSWORD']}@host:5432/shop")
df_orders = pd.read_sql_table('orders', _engine_1)

Each run also snapshots the input to parquet for reproducibility. MongoDB sources use collection selection (no custom query).

Tips & common mistakes

  • Passwords are never embedded. Exported code resolves the secret at runtime from its reference scheme โ€” os.environ[...] for a bare name or env:NAME, keyring.get_password(...) for keyring:NAME (recommended on desktop), or a file:/path read for a mounted secret file. See Connections for the full scheme list.
  • Test the connection first. Use the connection's Test action to confirm credentials and reachability before wiring it into a flow.

See also