Skip to content

armgit / moss

Public Imported from GitHub
0

MOSS: a small, mostly 3D-printed litter-picking rover. Cleaner streets, one maker at a time.

Python
6.9 KB
Raw Blame

Litter: see it, go to it, pick it

moss-litter is moss-explore plus two modules: the litter detection on the nose camera and the mission that drives from a sighting to the pick. The arm runs in its own dimos run (moss-arm, docs/arm.md); the mission calls its cycle over RPC.

.venv/bin/dimos run moss-dimos.moss-arm        # one terminal: the arm's coordinator
EXPLORE=1 tools/moss-fly.sh litter             # another: the gates, then explore, detect, approach, pick, explore

tools/moss-fly.sh litter runs the same gates and proofs as explore and keeps the arming: the mission does not send the first explore_cmd itself under the launcher (MOSS_MISSION_AUTOSTART=0). By hand, .venv/bin/dimos run moss-dimos.moss-litter starts exploring on its own.

Detection

moss_dimos/litter_detect.py is dimOS's object scene registration (ObjectSceneRegistrationModule) with one change: it runs a YOLO trained on litter from a weights file of ours instead of the open-vocabulary model dimOS downloads. The module aligns colour and depth frames, lifts every box to a 3D object through the depth image and the camera's transform to the world frame of the odometry, and publishes on objects the ones confirmed by repeated sightings (MOSS_LITTER_MIN_DETECTIONS, 6). Two sightings closer than 0.2 m are one object.

The weights are not in this repository. The ones the mission is tuned to are a small YOLO trained on waste images with five material classes (glass, metal, paper, plastic, waste), published under the MIT licence at github.com/gianlucasposito/YOLO-Waste-Detection (best_model.pt, 6 MB):

mkdir -p ~/.local/state/moss/models
curl -L -o ~/.local/state/moss/models/litter.pt \
  https://raw.githubusercontent.com/gianlucasposito/YOLO-Waste-Detection/main/best_model.pt

MOSS_LITTER_WEIGHTS points elsewhere; MOSS_LITTER_CLASSES names the classes the mission picks up (the five by default); MOSS_LITTER_DEVICE forces cpu or cuda. The detector needs torch and ultralytics in the robot's venv, which the base stack does not: on an arm64 host without a CUDA torch, the CPU wheels do (about 200 ms a frame for this model on the rover's CPU is the expectation; the camera runs at 5 fps):

uv pip install --python .venv/bin/python --index-url https://download.pytorch.org/whl/cpu torch torchvision
uv pip install --python .venv/bin/python ultralytics lap    # lap: the tracker ultralytics would otherwise fetch on first use

Measure the detector on the floor before trusting it

tools/moss_litter_probe.py runs the nose camera and the detector alone, stack down, and writes one CSV line per detection with the object's position in base_link (x forward, y left, z up, the lidar's floor projection: the frame of MOSS_PICK_XY_M), then a summary: the share of frames that saw the litter, the median position and its spread per class, the detector's time per frame, and, with --expect X Y, how far the median is from where the tape measure put the object.

tools/moss-fly.sh stop                                   # the camera must be free; the robot's venv has torch and ultralytics, the system python does not
.venv/bin/python tools/moss_litter_probe.py --seconds 10 --expect 1.00 0.00   # a can 1.00 m ahead of base_link's origin, on its x axis
.venv/bin/python tools/moss_litter_probe.py --fake --frames 20            # no camera, no torch: the chain on a synthetic frame

Three or four pieces of litter at measured places, ten seconds each, give the two numbers this chain needs before any training: the hit rate on our floor and the position error at the approach distance. The geometry is the one the stack uses: the tool reads <MOSS_STATE_DIR>/robot.env first, as the launcher does, so MOSS_CAMERA_XYZ_M and MOSS_CAMERA_PITCH_DEG set there are the ones it runs with (the shell's own values win, --camera-xyz and --pitch-deg win over both; the first line it prints says which file and which keys it took). The CSV lands in <MOSS_STATE_DIR>/litter_probe/.

The mission

moss_dimos/mission.py. The decision (MissionLogic) is pure and the bench drives it on a fake robot; the module around it reads objects, odom and goal_reached, writes goal_request, tele_cmd_vel, stop_explore_cmd, explore_cmd and a mission_state line, ticks at 10 Hz.

State What happens
EXPLORING the explorer drives; a confirmed object of a litter class within 4 m, not picked or skipped, becomes the target
STOPPING stop_explore_cmd, then 3 s for the explorer's own "stay here" goal to land first
APPROACHING goal_request: 0.6 m before the object on the line from the robot, facing it; goal_reached True → SERVOING, False → the object is skipped; 90 s → SERVOING from wherever the rover is
SERVOING tele_cmd_vel at every tick (the pad's channel: the movement manager lets it win): one law drives the robot's pick point onto the object (the pick point is the controlled point: w = vy / px, v = vx + w py), within 2 cm; at most 0.08 m/s and 0.4 rad/s. The base raises any non-zero command to its power floor and pivots at full power, so within MOSS_SERVO_FAR_M (0.25) the twist comes in pulses: one tick on (MOSS_SERVO_PULSE_S, 0.1), MOSS_SERVO_REST_S (0.4) of zeros, a step at a time; farther it is continuous. 60 s → skipped
PICKING the pick command (MOSS_PICK_CMD, default tools/moss_arm_play.py; MOSS_PICK_SIM=1 adds --sim; none marks the object picked without moving the arm); exit 0 → picked, else skipped
RESUMING explore_cmd, then EXPLORING; a picked object, and anything within 0.15 m of it, is never a target again; a skipped one is tried again after 10 min

The pick point is where the cycle's pick pose puts the tool, in base_link (the lidar's floor projection): MOSS_PICK_XY_M, default 0.215 -0.094 (the cycle's tool point 0.345 -0.010 less the scanner's axis 0.130 0.084, both in the CAD scene's chassis coordinates). The camera's mount must use the same origin: with base_link at the scanner, MOSS_CAMERA_XYZ_M="0.026 -0.084 0.075" (the CAD nose position less the scanner's axis; the profile's default is the CAD nose position itself). The whole final approach rests on these; a tape measure from the lidar's axis to the arm's pan axis on the rover is the number to set before a run. MOSS_STANDOFF_M (0.6) and MOSS_PICK_TOL_M (0.02) are the other two knobs. MOSS_MISSION_AUTOSTART=0 leaves the first explore_cmd to the cockpit.

python benches/bench_moss_mission_cold.py      # the logic on a fake robot; the module under dimOS

What this does not do yet

The approach is open-loop on odometry once the object is confirmed: the object's world position is refreshed while the camera still sees it, and the last 20 cm run on the odometry alone. The pick pose is the cycle's one pose: the rover brings the object to it, the arm does not reach for the object. The bin is the rover's own. No wrist-camera check before the grasp.