2HoursProject/vision-alignment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
cpu
2026-09-04 15:48:02 +02:00
commit 64b4982385
10 changed files with 2394 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Readme.md
+242
View File
@@ -0,0 +1,242 @@
# gSender Vision Alignment Plugin (FluidNC)
## Overview
The **Vision Alignment Plugin** is a custom extension for gSender (v1.7.0-Edge-1 and later) running on **Linux**. It interfaces with FluidNC-based CNC controllers and uses a live camera feed with a digital cross-hair overlay for highly accurate stock alignment.
By guiding the user to locate three known fiducial markers on their stock, the plugin mathematically determines the exact X/Y workspace origin and the angular deviation of the material. Since FluidNC does not natively support hardware-level coordinate rotation (`G68`), the plugin automatically sets the work zero and **transforms the loaded G-code in software**—applying a rotation matrix to compensate for the skewed stock—completely eliminating the need for perfect physical stock alignment.
## Features
* **Live Camera Feed:** Streams real-time video directly within the gSender interface using any standard USB webcam or microscope camera.
* **Software Cross-Hair Overlay:** Provides a customizable, digital center-target for high-precision visual probing.
* **Three-Point Fiducial Registration:**
* Captures machine coordinates at three distinct points on the stock.
* Point 1 establishes the reference origin.
* Points 2 and 3 establish the true X/Y axes vectors and verify scale/skew.
* **Software-Side Skew Compensation:** Calculates the exact rotational angle (θ) of the stock. The plugin then parses your loaded G-code and applies a rotational matrix (adjusting all `X`, `Y`, `I`, and `J` values) to dynamically match the skew of your stock before sending the job to the controller.
* **Automated Work Zeroing:** Automatically calculates and applies camera-to-spindle offsets and issues `G10 L20` commands to establish the new Work Coordinate System (WCS).
## Prerequisites
* **gSender:** v1.7.0-Edge-1 or later on Linux (plugin system required).
* **Controller:** A CNC machine running **FluidNC** firmware (or any Grbl / grblHAL-based firmware).
* **Hardware:** A rigidly mounted camera (e.g., attached to the spindle or Z-axis carriage) with calibrated X/Y offsets.
* **Development Environment:** Node.js 18+ and yarn (or npm).
## Installation (Linux end users)
gSender loads plugins from:
```text
~/.config/gSender/plugins/
```
(The exact path is also shown under **Tools → Plugins** inside the app.)
1. Obtain a **built** plugin folder (one that already contains `gsender-plugin.json` **and** a `ui/` directory with `index.html`).
2. Copy that folder into `~/.config/gSender/plugins/`, **or** use **Tools → Plugins → Import** and select the folder.
3. Restart gSender. The plugin appears on the **Tools** page.
> Plugins with a `com.sienci.*` id are labelled “Sienci official”; all others appear as “Community”.
## Usage Guide
1. Open the **Tools** page and launch **Vision Alignment**.
2. Select your camera device from the dropdown to initialize the feed.
3. **Calibrate Camera Offset (if not already done):** Enter the physical X and Y distance between your spindle center and camera center.
4. **Point 1 (Origin):** Jog the machine until the cross-hair is perfectly centered on your first fiducial. Click **"Mark Point 1"**.
5. **Point 2 (Axis Reference):** Jog to the second fiducial (defines your primary axis line) and click **"Mark Point 2"**.
6. **Point 3 (Verification):** Jog to the third fiducial and click **"Mark Point 3"**.
7. Click **"Align & Zero Workspace"**.
* The plugin processes the affine transformation math.
* It zeros the workspace to the calculated origin point.
* It applies a rotation matrix to the loaded G-code so the toolpath matches the rotational skew of your stock.
---
## Plugin architecture (gSender 1.7+ on Linux)
gSender plugins are self-contained SPA folders. Each plugin **must** contain:
* `gsender-plugin.json` – manifest
* `ui/` – **built** frontend (produced by Vite; not present until you build)
### Manifest (`gsender-plugin.json`)
| Field | Required | Description |
|--------------------|----------|-------------|
| `id` | yes | Unique reverse-DNS id (e.g. `com.yourname.vision-alignment`). |
| `name` | yes | Display name shown on the Tools page. |
| `version` | yes | Semver string. |
| `description` | no | Short blurb on the plugin card. |
| `engine` | no | Compatible gSender version range (e.g. `>=1.7.0`). |
| `ui.entry` | yes | Path to the built entry HTML (usually `ui/index.html`). |
| `ui.contributions` | no | Array of `{ slot, route, label }` – most plugins use `"slot": "tools-page"`. |
| `capabilities` | no | Bridge permissions the plugin is allowed to use. |
Example skeleton:
```json
{
"id": "com.yourname.vision-alignment",
"name": "Vision Alignment",
"description": "Camera-based stock alignment and software skew compensation for FluidNC.",
"version": "0.1.0",
"engine": ">=1.7.0",
"ui": {
"entry": "ui/index.html",
"contributions": [
{
"slot": "tools-page",
"route": "vision-alignment",
"label": "Vision Alignment"
}
]
},
"capabilities": {
"requestTypes": ["gcode:load:to:visualizer"],
"topics": ["workspace"],
"allowedFunctions": ["gcode", "machine", "useWorkspaceState"]
}
}
```
The bridge **denies** any call the plugin was not granted.
### Official examples & SDK
Official examples live in the gSender repo under [`plugins/`](https://github.com/Sienci-Labs/gsender/tree/v1.7.0-Edge-1/plugins):
| Folder | Stack | Demonstrates |
|-------------------|--------------------------------|--------------|
| `example-hello/` | Plain JS + Vite | Bridge client, subscriptions |
| `react-ts-app/` | React + TypeScript + Vite | React hooks |
| `example-viewer/` | Plain JS + Vite | Embedded G-code preview |
| `basic-cam/` | React + TypeScript + Vite + Tailwind | Full reference CAM plugin |
SDK package: `@sienci/gsender-plugin-sdk` (source lives in `packages/plugin-sdk`).
| Import | Purpose |
|--------|---------|
| `@sienci/gsender-plugin-sdk` | Framework-agnostic bridge |
| `@sienci/gsender-plugin-sdk/react` | React hooks |
| `@sienci/gsender-plugin-sdk/viewer` | G-code viewer (`@sienci/gviewer`) |
| `@sienci/gsender-plugin-sdk/vite` | Vite plugin for correct externalisation & import map |
Always build plugins with the SDK’s Vite helper:
```ts
// vite.config.ts
import gsenderPlugin from "@sienci/gsender-plugin-sdk/vite";
import react from "@vitejs/plugin-react";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [react(), gsenderPlugin()],
base: "./",
build: { outDir: "ui", emptyOutDir: true },
});
```
Dark mode is applied by gSender as the class `html.dark` on the plugin iframe—use class-based dark styles (or Tailwind `dark:` with class strategy), not `prefers-color-scheme`.
---
## Building plugins (Linux development)
### Important: build the SDK first
The example plugins depend on the local SDK via `file:../../packages/plugin-sdk`. That package’s `package.json` points at `./dist/…`, which **does not exist until you build it**. If you skip this step you will see:
```text
Failed to resolve entry for package "@sienci/gsender-plugin-sdk"
```
### Step-by-step
From the **gSender repo root** (`~/proj/gsender` or similar):
```bash
# 1. Build the plugin SDK (creates packages/plugin-sdk/dist/)
cd packages/plugin-sdk
yarn install
yarn build
# 2. Build the example plugin(s) you need
cd ../../plugins/example-hello
yarn install
yarn build # creates ui/index.html
# Repeat for other examples if desired:
# cd ../react-ts-app && yarn install && yarn build
# cd ../example-viewer && yarn install && yarn build
# cd ../basic-cam && yarn install && yarn build
```
Or build everything in one go:
```bash
cd ~/proj/gsender
(cd packages/plugin-sdk && yarn install && yarn build)
for d in plugins/example-hello plugins/react-ts-app plugins/example-viewer plugins/basic-cam; do
echo "=== Building $d ==="
(cd "$d" && yarn install && yarn build)
done
```
After a successful build you should have:
```text
plugins/example-hello/ui/index.html
```
### “UI entry not found: ui/index.html”
This red message in **Tools → Plugins** means the plugin folder exists but has not been built (no `ui/` directory). Run `yarn build` inside the plugin folder (after the SDK is built) and refresh/restart gSender.
### Watch mode while developing
```bash
cd plugins/example-hello # or your own plugin
yarn build -- --watch
```
gSender (when started with `yarn dev` / `NODE_ENV=development`) watches each plugin’s `ui/` directory and reloads the iframe automatically.
### Starting from an official template
1. Copy the closest example (`example-hello`, `react-ts-app`, `example-viewer`, or `basic-cam`).
2. Edit `gsender-plugin.json` – change `id`, `name`, `description`, `route`, and `label`.
3. Rename the folder if desired (the manifest `id` is what matters).
4. Ensure the SDK is built (`packages/plugin-sdk` → `yarn build`).
5. `yarn install && yarn build` inside the new plugin folder.
6. Place the folder in `~/.config/gSender/plugins/` **or** keep it under the gSender repo’s `plugins/` for local dev.
### Local development inside the gSender source tree
When gSender runs in development (`NODE_ENV=development`, e.g. `yarn dev` or `yarn electron:hot`):
* Plugins are loaded from **both** the repo’s `plugins/` folder **and** `~/.config/gSender/plugins/`.
* Repo plugins take precedence when ids collide.
* Extra search paths: `GSENDER_PLUGINS_DIRS` (colon-separated on Linux).
* Adding a brand-new plugin folder requires a one-time server restart; subsequent edits hot-reload once `ui/` is present.
---
## Development & Testing Workflow
### Leveraging the FluidNC PC Simulation Feature
When developing a CNC plugin that manipulates Work Coordinate Systems and modifies G-code on the fly, testing on a physical machine carries a high risk of accidental crashes. **Because you are developing on Linux, the FluidNC PC Simulation feature is highly recommended.**
FluidNC provides a **PC port/simulation mode** that compiles and runs the firmware as a native executable, completely bypassing the need for an ESP32 microcontroller or physical steppers.
#### Why it is critical for this plugin’s development
1. **Zero Hardware Risk:** Safely test the trigonometry for software-side G-code transformation (linear moves and arcs) and workspace zeroing (`G10 L20`) without risk of plunging a real endmill into the spoilboard.
2. **Rapid Iteration:** Run gSender Edge and the FluidNC simulator side-by-side on the same Linux machine—no workshop computer or hardware cabling required for UI/math work.
3. **Toolpath Verification:** The simulator behaves identically to real FluidNC firmware. Send the dynamically rotated G-code from the plugin and query the controller (`?` or `$#`) to confirm the virtual machine executes the skewed toolpath correctly.
#### Setup Guide for the Simulator
See the [Development](Development.md) guide.
## License
MIT License. See [LICENSE](LICENSE) for more information.