All projects

AI & ML

RecallAI

A visual memory assistant

Helping AI remember where you last saw things.

Role
Temporal Memory and backend integration.
Status
Hackathon prototype
Year
2026
Team
Built at Cal Hacks 13.0 in Berkeley with Kentstar Harsono, jgsam and Chawin.
Team repository (opens in a new tab)Devpost (opens in a new tab)
RecallAI's sighting history panel for a backpack, showing where it was recalled, how long ago, and a snapshot marked as seen and unchecked.
Sighting history for one belonging, cropped to the panel. Time-aware recall is the part I built.

01

The problem

"Where did I leave my keys?" is a memory problem. A camera that only detects objects can say what it sees now, but not what it saw twenty minutes ago, or when.

RecallAI watches through an ordinary webcam, recognises your own belongings rather than generic objects, and remembers where each one was last seen, with a photo as proof.

02

My contribution

I built the Temporal Memory layer: last-seen retrieval, historical observations, elapsed-time computation and the grounded, time-aware answers the assistant gives, plus backend integration.

Every answer is computed from stored sighting rows, never guessed. The camera and vision pipeline, the Grok-powered assistant, the live dashboard and the sync layer were built by the team.

  • Python
  • FastAPI
  • YOLO
  • DINOv2
  • SQLite

03

How it works

  1. Register

    Show the camera an item from three angles. There is no training step.

  2. Detect and recognise

    YOLO finds objects in each frame, then DINOv2 compares each one with your reference photos. A similarity of 0.70 or more means it is yours.

  3. Remember

    A sighting snapshot is saved at most every 15 seconds while an item is in view, and another 5 seconds after it leaves.

  4. Answer

    Time-aware retrieval turns the stored sightings into a grounded answer with links to the snapshots it used.

04

Engineering details

  • Observed, not inferred

    A sighting is where an item was. Nothing in the memory layer says where it is now. The last-seen time is the newest sighting, never the moment the item went missing.

  • Defensive time handling

    Timestamps are stored as UTC ISO strings, naive values are read as UTC, small clock jitter is clamped, and a stale 'seen' row means recording stopped, not that the item left.

  • Read-only and per user

    Looking an item up never creates a sighting or resets its age, and every query is scoped to the signed-in user's own rows.

  • Recognition by example

    The team picked DINOv2 over CLIP after it separated 'my bottle' from 'a bottle' 3 to 4 times better on their own frames, and set the 0.70 threshold from measurements.

05

Results & limitations

It works end to end on a laptop: register an item, watch it move around a room, and ask where it was last seen. The memory layer's answers are traceable to stored sightings, which is the property I cared about most.

  • 3 to 4x

    better separation than CLIP

    DINOv2 vs CLIP on the team's own frames, telling your item from a similar one.

  • 0.77 to 0.95

    similarity, same item

    Other objects scored 0.64 or lower, which is why the match threshold is 0.70.

  • 250+

    backend tests

    With the assistant mocked, and CI running on every push.

Limitations and honest caveats

  • A hackathon prototype tested on a handful of items in one venue.
  • Sightings are where an item was seen, not a live location. It does not do continuous perfect tracking.
  • The conversational assistant needs an xAI API key. Without one, everything except the assistant still works.
  • Setup is Windows-first, and the repository belongs to a teammate.