Infer defect location from photos and dictation

Objective

  • Auto-fill defect-level location on comments (e.g. utility room, roof, master bedroom) by combining photo context and spoken narrative, so inspectors stop double-entering location that is already obvious from media or dictation.
  • Keep scope on the defect’s real-world location, not the report’s section / subsection tree—that stays a separate concern.

Background

  • May 15 feature sync (Chris): Server-side gap in Report Writer: photos already carry some location context, and inspectors say the location aloud, but the comment location field is still managed manually.
  • Desired inputs: (1) locations tied to photos, (2) what the inspector says into the phone—example: “water stain, master bedroom” should land the comment location as master bedroom.
  • Decision needed: confidence thresholds, override UX when inference is wrong, and privacy / retention for audio snippets.

Scope

Report Writer product (primary)

  • Decision needed: concrete repo / branch / package where Report Writer comment, photo, and audio models live in this workspace snapshot those paths were not found under obvious names—coordinate with owners of ATT-1672 (server schemas) and the mobile MVP repo.

Backend (conceptual)

  • Multimodal pipeline (image metadata + ASR transcript + LLM or rules) and persistence onto the comment location field—Decision needed whether inference runs sync on save or async job with spinner.

Mobile

  • Dictation capture quality, background noise, and offline queue behavior—Decision needed which SDK path the Report Writer app uses.

Related client portal (context only)

  • attik-frontend client report surfaces (e.g. defect presentation under src/app/client/reports/) are consumer UIs for finished reports, not the Report Writer authoring app—cite only if product wants labeling consistency between authored location and published report.

References

  • Sibling planning: ATT-1672 (move Report Writer schemas to backend).
  • Client report presentation tree (naming alignment only): attik-frontend/src/app/client/reports/ (e.g. components/StylesAndMaterialsCard.tsx lives alongside other report UI).

Please authenticate to join the conversation.

Upvoters
Status

In Progress

Board
💻

Report Writer

Date

15 days ago

Author

Linear

Subscribe to post

Get notified by email when there are changes.