243 lines
11 KiB
Markdown
243 lines
11 KiB
Markdown
# gSender Vision Alignment Plugin (FluidNC)
|
||
|
||
## Overview
|
||
The **Vision Alignment Plugin** is a custom extension for gSender (v1.7.0-Edge-1 and later) running on **Linux**. It interfaces with FluidNC-based CNC controllers and uses a live camera feed with a digital cross-hair overlay for highly accurate stock alignment.
|
||
|
||
By guiding the user to locate three known fiducial markers on their stock, the plugin mathematically determines the exact X/Y workspace origin and the angular deviation of the material. Since FluidNC does not natively support hardware-level coordinate rotation (`G68`), the plugin automatically sets the work zero and **transforms the loaded G-code in software**—applying a rotation matrix to compensate for the skewed stock—completely eliminating the need for perfect physical stock alignment.
|
||
|
||
## Features
|
||
* **Live Camera Feed:** Streams real-time video directly within the gSender interface using any standard USB webcam or microscope camera.
|
||
* **Software Cross-Hair Overlay:** Provides a customizable, digital center-target for high-precision visual probing.
|
||
* **Three-Point Fiducial Registration:**
|
||
* Captures machine coordinates at three distinct points on the stock.
|
||
* Point 1 establishes the reference origin.
|
||
* Points 2 and 3 establish the true X/Y axes vectors and verify scale/skew.
|
||
* **Software-Side Skew Compensation:** Calculates the exact rotational angle (θ) of the stock. The plugin then parses your loaded G-code and applies a rotational matrix (adjusting all `X`, `Y`, `I`, and `J` values) to dynamically match the skew of your stock before sending the job to the controller.
|
||
* **Automated Work Zeroing:** Automatically calculates and applies camera-to-spindle offsets and issues `G10 L20` commands to establish the new Work Coordinate System (WCS).
|
||
|
||
## Prerequisites
|
||
* **gSender:** v1.7.0-Edge-1 or later on Linux (plugin system required).
|
||
* **Controller:** A CNC machine running **FluidNC** firmware (or any Grbl / grblHAL-based firmware).
|
||
* **Hardware:** A rigidly mounted camera (e.g., attached to the spindle or Z-axis carriage) with calibrated X/Y offsets.
|
||
* **Development Environment:** Node.js 18+ and yarn (or npm).
|
||
|
||
## Installation (Linux end users)
|
||
|
||
gSender loads plugins from:
|
||
|
||
```text
|
||
~/.config/gSender/plugins/
|
||
```
|
||
|
||
(The exact path is also shown under **Tools → Plugins** inside the app.)
|
||
|
||
1. Obtain a **built** plugin folder (one that already contains `gsender-plugin.json` **and** a `ui/` directory with `index.html`).
|
||
2. Copy that folder into `~/.config/gSender/plugins/`, **or** use **Tools → Plugins → Import** and select the folder.
|
||
3. Restart gSender. The plugin appears on the **Tools** page.
|
||
|
||
> Plugins with a `com.sienci.*` id are labelled “Sienci official”; all others appear as “Community”.
|
||
|
||
## Usage Guide
|
||
1. Open the **Tools** page and launch **Vision Alignment**.
|
||
2. Select your camera device from the dropdown to initialize the feed.
|
||
3. **Calibrate Camera Offset (if not already done):** Enter the physical X and Y distance between your spindle center and camera center.
|
||
4. **Point 1 (Origin):** Jog the machine until the cross-hair is perfectly centered on your first fiducial. Click **"Mark Point 1"**.
|
||
5. **Point 2 (Axis Reference):** Jog to the second fiducial (defines your primary axis line) and click **"Mark Point 2"**.
|
||
6. **Point 3 (Verification):** Jog to the third fiducial and click **"Mark Point 3"**.
|
||
7. Click **"Align & Zero Workspace"**.
|
||
* The plugin processes the affine transformation math.
|
||
* It zeros the workspace to the calculated origin point.
|
||
* It applies a rotation matrix to the loaded G-code so the toolpath matches the rotational skew of your stock.
|
||
|
||
---
|
||
|
||
## Plugin architecture (gSender 1.7+ on Linux)
|
||
|
||
gSender plugins are self-contained SPA folders. Each plugin **must** contain:
|
||
|
||
* `gsender-plugin.json` – manifest
|
||
* `ui/` – **built** frontend (produced by Vite; not present until you build)
|
||
|
||
### Manifest (`gsender-plugin.json`)
|
||
|
||
| Field | Required | Description |
|
||
|--------------------|----------|-------------|
|
||
| `id` | yes | Unique reverse-DNS id (e.g. `com.yourname.vision-alignment`). |
|
||
| `name` | yes | Display name shown on the Tools page. |
|
||
| `version` | yes | Semver string. |
|
||
| `description` | no | Short blurb on the plugin card. |
|
||
| `engine` | no | Compatible gSender version range (e.g. `>=1.7.0`). |
|
||
| `ui.entry` | yes | Path to the built entry HTML (usually `ui/index.html`). |
|
||
| `ui.contributions` | no | Array of `{ slot, route, label }` – most plugins use `"slot": "tools-page"`. |
|
||
| `capabilities` | no | Bridge permissions the plugin is allowed to use. |
|
||
|
||
Example skeleton:
|
||
|
||
```json
|
||
{
|
||
"id": "com.yourname.vision-alignment",
|
||
"name": "Vision Alignment",
|
||
"description": "Camera-based stock alignment and software skew compensation for FluidNC.",
|
||
"version": "0.1.0",
|
||
"engine": ">=1.7.0",
|
||
"ui": {
|
||
"entry": "ui/index.html",
|
||
"contributions": [
|
||
{
|
||
"slot": "tools-page",
|
||
"route": "vision-alignment",
|
||
"label": "Vision Alignment"
|
||
}
|
||
]
|
||
},
|
||
"capabilities": {
|
||
"requestTypes": ["gcode:load:to:visualizer"],
|
||
"topics": ["workspace"],
|
||
"allowedFunctions": ["gcode", "machine", "useWorkspaceState"]
|
||
}
|
||
}
|
||
```
|
||
|
||
The bridge **denies** any call the plugin was not granted.
|
||
|
||
### Official examples & SDK
|
||
|
||
Official examples live in the gSender repo under [`plugins/`](https://github.com/Sienci-Labs/gsender/tree/v1.7.0-Edge-1/plugins):
|
||
|
||
| Folder | Stack | Demonstrates |
|
||
|-------------------|--------------------------------|--------------|
|
||
| `example-hello/` | Plain JS + Vite | Bridge client, subscriptions |
|
||
| `react-ts-app/` | React + TypeScript + Vite | React hooks |
|
||
| `example-viewer/` | Plain JS + Vite | Embedded G-code preview |
|
||
| `basic-cam/` | React + TypeScript + Vite + Tailwind | Full reference CAM plugin |
|
||
|
||
SDK package: `@sienci/gsender-plugin-sdk` (source lives in `packages/plugin-sdk`).
|
||
|
||
| Import | Purpose |
|
||
|--------|---------|
|
||
| `@sienci/gsender-plugin-sdk` | Framework-agnostic bridge |
|
||
| `@sienci/gsender-plugin-sdk/react` | React hooks |
|
||
| `@sienci/gsender-plugin-sdk/viewer` | G-code viewer (`@sienci/gviewer`) |
|
||
| `@sienci/gsender-plugin-sdk/vite` | Vite plugin for correct externalisation & import map |
|
||
|
||
Always build plugins with the SDK’s Vite helper:
|
||
|
||
```ts
|
||
// vite.config.ts
|
||
import gsenderPlugin from "@sienci/gsender-plugin-sdk/vite";
|
||
import react from "@vitejs/plugin-react";
|
||
import { defineConfig } from "vite";
|
||
|
||
export default defineConfig({
|
||
plugins: [react(), gsenderPlugin()],
|
||
base: "./",
|
||
build: { outDir: "ui", emptyOutDir: true },
|
||
});
|
||
```
|
||
|
||
Dark mode is applied by gSender as the class `html.dark` on the plugin iframe—use class-based dark styles (or Tailwind `dark:` with class strategy), not `prefers-color-scheme`.
|
||
|
||
---
|
||
|
||
## Building plugins (Linux development)
|
||
|
||
### Important: build the SDK first
|
||
|
||
The example plugins depend on the local SDK via `file:../../packages/plugin-sdk`. That package’s `package.json` points at `./dist/…`, which **does not exist until you build it**. If you skip this step you will see:
|
||
|
||
```text
|
||
Failed to resolve entry for package "@sienci/gsender-plugin-sdk"
|
||
```
|
||
|
||
### Step-by-step
|
||
|
||
From the **gSender repo root** (`~/proj/gsender` or similar):
|
||
|
||
```bash
|
||
# 1. Build the plugin SDK (creates packages/plugin-sdk/dist/)
|
||
cd packages/plugin-sdk
|
||
yarn install
|
||
yarn build
|
||
|
||
# 2. Build the example plugin(s) you need
|
||
cd ../../plugins/example-hello
|
||
yarn install
|
||
yarn build # creates ui/index.html
|
||
|
||
# Repeat for other examples if desired:
|
||
# cd ../react-ts-app && yarn install && yarn build
|
||
# cd ../example-viewer && yarn install && yarn build
|
||
# cd ../basic-cam && yarn install && yarn build
|
||
```
|
||
|
||
Or build everything in one go:
|
||
|
||
```bash
|
||
cd ~/proj/gsender
|
||
|
||
(cd packages/plugin-sdk && yarn install && yarn build)
|
||
|
||
for d in plugins/example-hello plugins/react-ts-app plugins/example-viewer plugins/basic-cam; do
|
||
echo "=== Building $d ==="
|
||
(cd "$d" && yarn install && yarn build)
|
||
done
|
||
```
|
||
|
||
After a successful build you should have:
|
||
|
||
```text
|
||
plugins/example-hello/ui/index.html
|
||
```
|
||
|
||
### “UI entry not found: ui/index.html”
|
||
|
||
This red message in **Tools → Plugins** means the plugin folder exists but has not been built (no `ui/` directory). Run `yarn build` inside the plugin folder (after the SDK is built) and refresh/restart gSender.
|
||
|
||
### Watch mode while developing
|
||
|
||
```bash
|
||
cd plugins/example-hello # or your own plugin
|
||
yarn build -- --watch
|
||
```
|
||
|
||
gSender (when started with `yarn dev` / `NODE_ENV=development`) watches each plugin’s `ui/` directory and reloads the iframe automatically.
|
||
|
||
### Starting from an official template
|
||
|
||
1. Copy the closest example (`example-hello`, `react-ts-app`, `example-viewer`, or `basic-cam`).
|
||
2. Edit `gsender-plugin.json` – change `id`, `name`, `description`, `route`, and `label`.
|
||
3. Rename the folder if desired (the manifest `id` is what matters).
|
||
4. Ensure the SDK is built (`packages/plugin-sdk` → `yarn build`).
|
||
5. `yarn install && yarn build` inside the new plugin folder.
|
||
6. Place the folder in `~/.config/gSender/plugins/` **or** keep it under the gSender repo’s `plugins/` for local dev.
|
||
|
||
### Local development inside the gSender source tree
|
||
|
||
When gSender runs in development (`NODE_ENV=development`, e.g. `yarn dev` or `yarn electron:hot`):
|
||
|
||
* Plugins are loaded from **both** the repo’s `plugins/` folder **and** `~/.config/gSender/plugins/`.
|
||
* Repo plugins take precedence when ids collide.
|
||
* Extra search paths: `GSENDER_PLUGINS_DIRS` (colon-separated on Linux).
|
||
* Adding a brand-new plugin folder requires a one-time server restart; subsequent edits hot-reload once `ui/` is present.
|
||
|
||
---
|
||
|
||
## Development & Testing Workflow
|
||
|
||
### Leveraging the FluidNC PC Simulation Feature
|
||
|
||
When developing a CNC plugin that manipulates Work Coordinate Systems and modifies G-code on the fly, testing on a physical machine carries a high risk of accidental crashes. **Because you are developing on Linux, the FluidNC PC Simulation feature is highly recommended.**
|
||
|
||
FluidNC provides a **PC port/simulation mode** that compiles and runs the firmware as a native executable, completely bypassing the need for an ESP32 microcontroller or physical steppers.
|
||
|
||
#### Why it is critical for this plugin’s development
|
||
1. **Zero Hardware Risk:** Safely test the trigonometry for software-side G-code transformation (linear moves and arcs) and workspace zeroing (`G10 L20`) without risk of plunging a real endmill into the spoilboard.
|
||
2. **Rapid Iteration:** Run gSender Edge and the FluidNC simulator side-by-side on the same Linux machine—no workshop computer or hardware cabling required for UI/math work.
|
||
3. **Toolpath Verification:** The simulator behaves identically to real FluidNC firmware. Send the dynamically rotated G-code from the plugin and query the controller (`?` or `$#`) to confirm the virtual machine executes the skewed toolpath correctly.
|
||
|
||
#### Setup Guide for the Simulator
|
||
See the [Development](Development.md) guide.
|
||
|
||
## License
|
||
MIT License. See [LICENSE](LICENSE) for more information.
|