Skip to main content

Observability and Debugging

These executors record queries, export metrics, or support debugging and tests. Evaluate hot-path cost and data retention.

learn_domain​

Experimental Plugin (Introduced in v1.2.0)

learn_domain was first introduced in v1.2.0 and is currently in an experimental stage. Filtering rules, write trigger timing, and the integration interface with dynamic_domain_set may change in future releases. Before deploying in production, please evaluate the impact of learning rates and disk writes on the persistence layer.

Purpose​

Observes qnames in the DNS pipeline and writes matching request names into a target dynamic_domain_set. It is a side-effect executor and never modifies the DNS request or response.

Example Configuration​

- tag: learned_allow
  type: dynamic_domain_set
  args:
    path: "/etc/oxidns/learned-allow.txt"

- tag: learn_allow
  type: learn_domain
  args:
    provider: learned_allow
    phase: after
    questions: first
    qtypes: ["A", "AAAA"]
    success_only: true
    answer_required: true
    rule_kind: full
    async: true
    error_mode: continue
    timeout: 1s

Configuration Details​

provider​

  • Type: string; Required: yes
  • Purpose: Target dynamic_domain_set provider tag.
  • Constraint: must reference dynamic_domain_set, not a regular domain_set.

phase​

  • Type: string; Required: no; Default: after
  • Values: before, after
  • Purpose:
    • before: learn from the request questions before downstream executors run.
    • after: run downstream first, then decide whether to learn based on the response.

questions​

  • Type: string; Required: no; Default: first
  • Values: first, all
  • Purpose: Learns only the first question or all questions.

qtypes​

  • Type: array; Required: no; Default: ["A", "AAAA"]
  • Purpose: DNS query types eligible for learning.

success_only​

  • Type: bool; Required: no; Default: true
  • Purpose: Only applies to phase: after; requires NOERROR.

answer_required​

  • Type: bool; Required: no; Default: true
  • Purpose: Only applies to phase: after; requires at least one answer record.

rule_kind​

  • Type: string; Required: no; Default: full
  • Values: full, domain
  • Purpose: Rule kind written into the target dynamic_domain_set.

async​

  • Type: bool; Required: no; Default: true
  • Purpose: Enqueues learned rules and continues immediately. When false, waits for provider persistence.

error_mode​

  • Type: string; Required: no; Default: continue
  • Values: continue, stop, fail
  • Purpose: Controls execution behavior if learning fails.

timeout​

  • Type: duration; Required: no; Default: 1s
  • Purpose: Limits provider write waiting time when async: false.

Behavior​

  • Learned domains are normalized to lowercase with trailing dots removed.
  • The default output is full:example.com to avoid broad accidental matches.
  • With the default phase: after, only successful A / AAAA responses with answers are learned.
  • phase: before does not inspect response conditions.
  • v1 only learns the request question qname and does not learn CNAME targets.

Typical Uses​

  • Add successfully resolved domains to a dynamic allow list.
  • Persist domains seen on a policy branch into a local file for later qname $learned_allow or parent domain_set matching.

query_summary​

Purpose​

Records concise query summaries.

Example Configuration​

- tag: summary_main
  type: query_summary
  args:
    # Extra title so multiple summary points are easy to distinguish
    msg: "main pipeline"

Configuration Details​

msg​

  • Type: string; Required: no
  • Purpose: Extra summary label.

quick setup​

- exec: "query_summary main"

Behavior​

  • Emits compact logs or summaries for operator visibility.

Typical Uses​

  • Light observability on the main path
  • Distinguish different branches

query_recorder​

Purpose​

Persists the entry request, the post-next response, and sequence execution-path events into a recorder-owned SQLite database, then exposes history, aggregate stats, and an SSE stream.

Example Configuration​

- tag: query_recorder_main
  type: query_recorder
  args:
    # SQLite path for this recorder. Multiple recorders may share one file.
    path: "./data/query-recorder-main.sqlite"
    # Hot-path enqueue buffer size
    queue_size: 8192
    # Batch size per SQLite flush
    batch_size: 256
    # Background flush interval in milliseconds
    flush_interval_ms: 200
    # Number of recent records kept in memory for SSE tail replay
    memory_tail: 1024
    # Retention window in days; minimum 1
    retention_days: 7
    # Cleanup interval in hours; minimum 1
    cleanup_interval_hours: 1
    # Maximum concurrent SQLite readers; minimum 1
    reader_concurrency: 2

Configuration Details​

path​

  • Type: string; Required: yes
  • Purpose: SQLite path for this recorder.

queue_size​

  • Type: integer; Required: no; Default: 8192
  • Purpose: Bounded queue size between the request path and the writer thread.

batch_size​

  • Type: integer; Required: no; Default: 256
  • Purpose: Number of records flushed per SQLite batch.

flush_interval_ms​

  • Type: integer; Required: no; Default: 200
  • Purpose: Maximum batch flush interval in milliseconds.

memory_tail​

  • Type: integer; Required: no; Default: 1024
  • Purpose: Size of the in-memory tail used by stream?tail=n.

retention_days​

  • Type: integer; Required: no; Default: 7; Minimum: 1
  • Purpose: Record retention window. Expired rows are deleted by the cleanup task.

cleanup_interval_hours​

  • Type: integer; Required: no; Default: 1; Minimum: 1
  • Purpose: Cleanup task cadence.

reader_concurrency​

  • Type: integer; Required: no; Default: 2; Minimum: 1
  • Purpose: Limits concurrent SQLite readers used by WebUI / API history and stats queries, so read bursts on large recorder databases do not occupy too many blocking threads or too much memory.

Behavior​

  • This is a pure executor observer and does not change server finalization logic.
  • It captures a structured snapshot of the entry request, enables DnsContext.execution_path, runs next, and commits immediately after next returns.
  • Successful runs store the current response. Failed runs store error and an empty response shape.
  • Request and response payloads are not stored as wire blobs. Question, RR, and EDNS fields are extracted into JSON text columns.
  • The client_ip field comes from the transport source address seen by OxiDNS; if a local forwarding chain such as systemd-resolved, dnsmasq, AdGuardHome, dae, or clash sits in front, rows may show only 127.0.0.1.
  • Each recorder uses versioned tables under the same prefix:
    • qr_<safe_tag>_<fnv64hex>_v1_records
    • qr_<safe_tag>_<fnv64hex>_v1_steps
    • qr_<safe_tag>_<fnv64hex>_v1_questions
    • qr_<safe_tag>_<fnv64hex>_v1_meta
  • records contains only the fixed schema fields for structured snapshots. steps stores sequence path events for path analysis and hit-rate reporting. questions is a derived index table from questions_json for faster qname/qtype queries. meta records completed derived-table migrations.
  • Every recorder owns its own bounded queue, SQLite connection, writer thread, tail buffer, and SSE broadcaster.
  • Recorders that share the same path coordinate readers, writers, and maintenance by normalized database path; tables and in-memory state remain isolated by tag.
  • Periodic retention cleanup deletes expired rows, reclaims every completely free page, and truncates the WAL last. The database file shrinks only when deletion produces complete free pages; small deletions may instead leave reusable space inside live pages.
  • If an existing database still uses auto_vacuum=NONE, its first periodic or manual cleanup performs one full VACUUM to migrate it to INCREMENTAL. This may temporarily pause recorder database reads and writes and require additional temporary disk space, but it does not block DNS request processing.
  • A maintenance failure does not stop the writer. Periodic cleanup retries on a later interval and structured logs report page counts, file sizes, and the failure stage.

Data Shape​

  • questions_json is always a question array, for example:
[
  { "name": "www.example.com.", "qtype": "A", "qclass": "IN" }
]
  • answers_json, authorities_json, additionals_json, and signature_json are RR arrays, for example:
[
  {
    "name": "www.example.com.",
    "class": "IN",
    "ttl": 300,
    "rr_type": "A",
    "payload_kind": "A",
    "payload_text": "192.0.2.1",
    "payload": { "ip": "192.0.2.1" }
  }
]
  • req_edns_json and resp_edns_json are EDNS objects or NULL.
  • The v1 suffix in the table name is the schema version. Future upgrades add new versioned tables rather than altering the existing ones in place.

API​

  • GET /plugins/<tag>/records
    • Returns record rows ordered by created_at_ms descending.
    • Query parameters:
      • cursor=<created_at_ms>:<id>
      • limit=<n>, default 100, max 500
      • since_ms=<unix_ms>
      • until_ms=<unix_ms>
      • qname=<text>, substring match against request question names
      • client_ip=<text>, substring match against the client IP string
      • qtype=<type> / rcode=<rcode> / status=all|error|has_response|no_response
  • GET /plugins/<tag>/records/<id>
    • Returns one full record plus steps.
  • DELETE /plugins/<tag>/records
    • Clears all history rows and steps for the current recorder, and clears the in-memory tail.
    • The operation first flushes the background writer queue, then deletes records in small auto-committed batches. It truncates the WAL after every deletion and page-reclamation batch, avoiding one transaction spanning the full clear operation. It returns cleared_records for the number of deleted main-table rows.
    • If history deletion succeeds but disk reclamation fails, the endpoint returns query_recorder_clear_failed with a message that history was already cleared; later maintenance can retry space reclamation.
  • GET /plugins/<tag>/stats/plugins
    • Returns hit stats grouped by matcher / executor / builtin.
    • Supports since_ms, until_ms, kind=matcher|executor|builtin|all, and the record filters.
  • GET /plugins/<tag>/stats/top_clients / stats/top_qnames
    • Returns client IP or QNAME rankings.
    • Supports limit=<n>, default 20; the backend no longer enforces a 200 cap.
    • Supports the same time range and filter parameters as records.
  • GET /plugins/<tag>/stats/qtype / stats/rcode
    • Returns QTYPE or RCODE distribution.
    • Supports the same time range and filter parameters as records.
  • GET /plugins/<tag>/stats/latency
    • Returns latency summary values, histogram buckets, and slow-query rankings.
    • Supports slow_limit=<n> or limit=<n>, default 20; the backend no longer enforces a 200 cap.
  • GET /plugins/<tag>/stats/timeseries
    • Returns query trends aggregated by minute or hour.
    • Supports bucket=minute|hour and buckets=<n> (default 60, maximum 720).
  • GET /plugins/<tag>/stream
    • Streams newly written records over SSE.
    • Supports tail=<n> to replay the in-memory tail.
    • Clients should send Accept: text/event-stream and tolerate heartbeat frames, error events, empty payloads, and brief disconnects.

Typical Uses​

  • Persistent audit and troubleshooting trails
  • sequence path analysis and plugin hit-rate reporting
  • Real-time query log feeds for dashboards or control planes
Notes
  • Place the recorder close to the entry point when the full main-path trace is required.
  • If an earlier branch short-circuits before the recorder, that request will not be recorded.
  • If next fails and the server later emits a fallback response, the database still reflects the plugin's point of view: error plus an empty response.
  • If the management API is disabled, the recorder still writes SQLite data but does not expose query or SSE routes.
  • To preserve real client IPs, point clients directly at OxiDNS or configure a trusted src_ip_header for HTTP/DoH reverse-proxy deployments.

metrics_collector​

Purpose​

Collects Prometheus metrics for query handling.

Example Configuration​

- tag: metrics_main
  type: metrics_collector
  args:
    # Collector label exported through /api/metrics
    name: "main"

Configuration Details​

name​

  • Type: string; Required: no
  • Purpose: Metrics label namespace.

quick setup​

- exec: "metrics_collector main"

Behavior​

  • Exposes query counters, inflight counts, and latency metrics through the management API.

API​

  • GET /api/metrics
    • Prometheus text format. This is the single global endpoint, and built-in metrics from other plugins are exported through the same route.

Typical Uses​

  • Prometheus integration
  • Observe multiple policy entry points separately

debug_print​

Purpose​

Prints a debug message.

Example Configuration​

- tag: debug_main
  type: debug_print
  args:
    # Log title; defaults to "debug print" when omitted
    msg: "before forward"

Configuration Details​

msg​

  • Type: string; Required: yes
  • Purpose: Message content.

quick setup​

- exec: "debug_print cache branch"

Typical Uses​

  • Temporary debugging
  • Reading sequence branches during development

sleep​

Purpose​

Sleeps for a bounded duration inside the chain.

Example Configuration​

- tag: sleep_100ms
  type: sleep
  args:
    # Add 100 ms of async delay
    duration: 100

Configuration Details​

duration​

  • Type: duration; Required: yes
  • Purpose: Sleep duration.

quick setup​

- exec: "sleep 100"

Typical Uses​

  • Testing
  • Timing experiments