vision-alignment/Readme.md

gSender Vision Alignment Plugin (FluidNC)

Overview

The Vision Alignment Plugin is a custom extension for gSender (v1.7.0-Edge-1 and later) running on Linux. It interfaces with FluidNC-based CNC controllers and uses a live camera feed with a digital cross-hair overlay for highly accurate stock alignment.

By guiding the user to locate three known fiducial markers on their stock, the plugin mathematically determines the exact X/Y workspace origin and the angular deviation of the material. Since FluidNC does not natively support hardware-level coordinate rotation (G68), the plugin automatically sets the work zero and transforms the loaded G-code in software—applying a rotation matrix to compensate for the skewed stock—completely eliminating the need for perfect physical stock alignment.

Features

• Live Camera Feed: Streams real-time video directly within the gSender interface using any standard USB webcam or microscope camera.
• Software Cross-Hair Overlay: Provides a customizable, digital center-target for high-precision visual probing.
• Three-Point Fiducial Registration:
  • Captures machine coordinates at three distinct points on the stock.
  • Point 1 establishes the reference origin.
  • Points 2 and 3 establish the true X/Y axes vectors and verify scale/skew.
• Software-Side Skew Compensation: Calculates the exact rotational angle (θ) of the stock. The plugin then parses your loaded G-code and applies a rotational matrix (adjusting all X, Y, I, and J values) to dynamically match the skew of your stock before sending the job to the controller.
• Automated Work Zeroing: Automatically calculates and applies camera-to-spindle offsets and issues G10 L20 commands to establish the new Work Coordinate System (WCS).

Prerequisites

• gSender: v1.7.0-Edge-1 or later on Linux (plugin system required).
• Controller: A CNC machine running FluidNC firmware (or any Grbl / grblHAL-based firmware).
• Hardware: A rigidly mounted camera (e.g., attached to the spindle or Z-axis carriage) with calibrated X/Y offsets.
• Development Environment: Node.js 18+ and yarn (or npm).

Installation (Linux end users)

gSender loads plugins from:

~/.config/gSender/plugins/

(The exact path is also shown under Tools → Plugins inside the app.)

1. Obtain a built plugin folder (one that already contains gsender-plugin.json and a ui/ directory with index.html).
2. Copy that folder into ~/.config/gSender/plugins/, or use Tools → Plugins → Import and select the folder.
3. Restart gSender. The plugin appears on the Tools page.

Plugins with a com.sienci.* id are labelled “Sienci official”; all others appear as “Community”.

Usage Guide

1. Open the Tools page and launch Vision Alignment.
2. Select your camera device from the dropdown to initialize the feed.
3. Calibrate Camera Offset (if not already done): Enter the physical X and Y distance between your spindle center and camera center.
4. Point 1 (Origin): Jog the machine until the cross-hair is perfectly centered on your first fiducial. Click "Mark Point 1".
5. Point 2 (Axis Reference): Jog to the second fiducial (defines your primary axis line) and click "Mark Point 2".
6. Point 3 (Verification): Jog to the third fiducial and click "Mark Point 3".
7. Click "Align & Zero Workspace".
   • The plugin processes the affine transformation math.
   • It zeros the workspace to the calculated origin point.
   • It applies a rotation matrix to the loaded G-code so the toolpath matches the rotational skew of your stock.

---

Plugin architecture (gSender 1.7+ on Linux)

gSender plugins are self-contained SPA folders. Each plugin must contain:

• gsender-plugin.json – manifest
• ui/ – built frontend (produced by Vite; not present until you build)

Manifest (gsender-plugin.json)

Field	Required	Description
id	yes	Unique reverse-DNS id (e.g. com.yourname.vision-alignment).
name	yes	Display name shown on the Tools page.
version	yes	Semver string.
description	no	Short blurb on the plugin card.
engine	no	Compatible gSender version range (e.g. >=1.7.0).
ui.entry	yes	Path to the built entry HTML (usually ui/index.html).
ui.contributions	no	Array of { slot, route, label } – most plugins use "slot": "tools-page".
capabilities	no	Bridge permissions the plugin is allowed to use.

Example skeleton:

{
  "id": "com.yourname.vision-alignment",
  "name": "Vision Alignment",
  "description": "Camera-based stock alignment and software skew compensation for FluidNC.",
  "version": "0.1.0",
  "engine": ">=1.7.0",
  "ui": {
    "entry": "ui/index.html",
    "contributions": [
      {
        "slot": "tools-page",
        "route": "vision-alignment",
        "label": "Vision Alignment"
      }
    ]
  },
  "capabilities": {
    "requestTypes": ["gcode:load:to:visualizer"],
    "topics": ["workspace"],
    "allowedFunctions": ["gcode", "machine", "useWorkspaceState"]
  }
}

The bridge denies any call the plugin was not granted.

Official examples & SDK

Official examples live in the gSender repo under plugins/:

Folder	Stack	Demonstrates
example-hello/	Plain JS + Vite	Bridge client, subscriptions
react-ts-app/	React + TypeScript + Vite	React hooks
example-viewer/	Plain JS + Vite	Embedded G-code preview
basic-cam/	React + TypeScript + Vite + Tailwind	Full reference CAM plugin

SDK package: @sienci/gsender-plugin-sdk (source lives in packages/plugin-sdk).

Import	Purpose
@sienci/gsender-plugin-sdk	Framework-agnostic bridge
@sienci/gsender-plugin-sdk/react	React hooks
@sienci/gsender-plugin-sdk/viewer	G-code viewer (@sienci/gviewer)
@sienci/gsender-plugin-sdk/vite	Vite plugin for correct externalisation & import map

Always build plugins with the SDK’s Vite helper:

// vite.config.ts
import gsenderPlugin from "@sienci/gsender-plugin-sdk/vite";
import react from "@vitejs/plugin-react";
import { defineConfig } from "vite";

export default defineConfig({
  plugins: [react(), gsenderPlugin()],
  base: "./",
  build: { outDir: "ui", emptyOutDir: true },
});

Dark mode is applied by gSender as the class html.dark on the plugin iframe—use class-based dark styles (or Tailwind dark: with class strategy), not prefers-color-scheme.

---

Building plugins (Linux development)

Important: build the SDK first

The example plugins depend on the local SDK via file:../../packages/plugin-sdk. That package’s package.json points at ./dist/…, which does not exist until you build it. If you skip this step you will see:

Failed to resolve entry for package "@sienci/gsender-plugin-sdk"

Step-by-step

From the gSender repo root (~/proj/gsender or similar):

# 1. Build the plugin SDK (creates packages/plugin-sdk/dist/)
cd packages/plugin-sdk
yarn install
yarn build

# 2. Build the example plugin(s) you need
cd ../../plugins/example-hello
yarn install
yarn build          # creates ui/index.html

# Repeat for other examples if desired:
# cd ../react-ts-app && yarn install && yarn build
# cd ../example-viewer && yarn install && yarn build
# cd ../basic-cam && yarn install && yarn build

Or build everything in one go:

cd ~/proj/gsender

(cd packages/plugin-sdk && yarn install && yarn build)

for d in plugins/example-hello plugins/react-ts-app plugins/example-viewer plugins/basic-cam; do
  echo "=== Building $d ==="
  (cd "$d" && yarn install && yarn build)
done

After a successful build you should have:

plugins/example-hello/ui/index.html

“UI entry not found: ui/index.html”

This red message in Tools → Plugins means the plugin folder exists but has not been built (no ui/ directory). Run yarn build inside the plugin folder (after the SDK is built) and refresh/restart gSender.

Watch mode while developing

cd plugins/example-hello   # or your own plugin
yarn build -- --watch

Create a prod package

cd gsender
git apply plugins/vision-alignment/allow_camera.patch
yarn build-prod
yarn start-electron

gSender (when started with yarn dev / NODE_ENV=development) watches each plugin’s ui/ directory and reloads the iframe automatically.

Starting from an official template

1. Copy the closest example (example-hello, react-ts-app, example-viewer, or basic-cam).
2. Edit gsender-plugin.json – change id, name, description, route, and label.
3. Rename the folder if desired (the manifest id is what matters).
4. Ensure the SDK is built (packages/plugin-sdk → yarn build).
5. yarn install && yarn build inside the new plugin folder.
6. Place the folder in ~/.config/gSender/plugins/ or keep it under the gSender repo’s plugins/ for local dev.

Local development inside the gSender source tree

When gSender runs in development (NODE_ENV=development, e.g. yarn dev or yarn electron:hot):

• Plugins are loaded from both the repo’s plugins/ folder and ~/.config/gSender/plugins/.
• Repo plugins take precedence when ids collide.
• Extra search paths: GSENDER_PLUGINS_DIRS (colon-separated on Linux).
• Adding a brand-new plugin folder requires a one-time server restart; subsequent edits hot-reload once ui/ is present.

---

Development & Testing Workflow

Leveraging the FluidNC PC Simulation Feature

When developing a CNC plugin that manipulates Work Coordinate Systems and modifies G-code on the fly, testing on a physical machine carries a high risk of accidental crashes. Because you are developing on Linux, the FluidNC PC Simulation feature is highly recommended.

FluidNC provides a PC port/simulation mode that compiles and runs the firmware as a native executable, completely bypassing the need for an ESP32 microcontroller or physical steppers.

Why it is critical for this plugin’s development

1. Zero Hardware Risk: Safely test the trigonometry for software-side G-code transformation (linear moves and arcs) and workspace zeroing (G10 L20) without risk of plunging a real endmill into the spoilboard.
2. Rapid Iteration: Run gSender Edge and the FluidNC simulator side-by-side on the same Linux machine—no workshop computer or hardware cabling required for UI/math work.
3. Toolpath Verification: The simulator behaves identically to real FluidNC firmware. Send the dynamically rotated G-code from the plugin and query the controller (? or $#) to confirm the virtual machine executes the skewed toolpath correctly.

Setup Guide for the Simulator

See the Development guide.

License

MIT License. See LICENSE for more information.