adeed manual processing using UVtools

Readme.md

@@ -6,19 +6,51 @@ KiCad PCB layers need to be converted to `.pm4n` files for direct UV exposure on
 
 ---
 
-## Setup
+## Preparation
 
-### 1. KiCad: Set your design grid
+### KiCad: Set your design grid
 In KiCad's PCB editor, go to `Preferences → Preferences → PCB Editor → Grids` and add a custom grid of 0.017 mm. This ensures trace edges land on pixel boundaries and avoids the sub-pixel rounding that causes the ±1px size error people see with arbitrary grids.
 
-### 2. Create the `Dummy.pm4n` file
+### Create the `Dummy.pm4n` file
 
-
+Open **CHITUBOX Basic** slicer (or any other slicer that works for your resin printer), select printer **Anycubic Photon Mono 4**, slice any tiny STL (e.g.: 1×1×0.05 mm box), and save as `Dummy.pm4n`.
-Open **CHITUBOX Basic** slicer (or any other slicer that works for your resin printer), select printer **Anycubic Photon Mono 4**, slice any tiny STL (e.g.: 1×1×0.05 mm box), and save as `Dummy.pm4n` in the same directory as `export.sh`.
 
 This file is reused for every job — it carries the correct LCD resolution metadata.
 
-### 3. Install dependencies
+## Manual processing
+
+`UVtools` has a dedicated `PCB Exposure` tool that converts a Gerber file to a pixel-perfect image given your printer's LCD resolution, specifically for exposing copper traces.
+
+### 1. Install `UVtools`
+
+```bash
+sudo apt-get install -y curl
+sudo bash -c "$(curl -fsSL https://raw.githubusercontent.com/sn4k3/UVtools/master/Scripts/install-dependencies.sh)"
+```
+
+### 2. Run `UVtools`
+
+```bash
+cd UVtools
+./UVtools
+```
+
+### 3. Export `.pm4n` file from Gerber
+
+- You export Gerber from KiCad (not SVG), which natively gives you positive/negative control per layer.
+- You open the `Dummy.pm4n` file in `UVtools` (a minimal valid `.pm4n` sliced by `Chitubox` or `Photon Workshop` with any tiny model), then use `Tools → PCB Exposure` to inject your Gerber layer e.g.: `Flow_Controller_Panel-Front.gbr`.
+- You control `inversion`, `invert` and the `exposure time` at the bottom of the `PCB exposure` dialog.
+- You save it as e.g.: `Flow_Controller_Panel-Front.pm4n` file and print it.
+
+![PCB_Exposure_Dialog_1](images/PCB_Exposure_Dialog_1.png)
+
+![PCB_Exposure_Dialog_2](images/PCB_Exposure_Dialog_2.png)
+
+---
+
+## Automated script
+
+### 1. Install dependencies
 
 ```bash
 python3 -m venv .venv
@@ -28,7 +60,7 @@ pip install pygerber Pillow numpy
 
 Or activate the `venv` once and put `source .venv/bin/activate` in your shell profile.
 
-### 4. Export multiple layers (e.g. copper + soldermask + silkscreen)
+### 2. Export multiple layers (e.g. copper + soldermask + silkscreen)
 
 ```bash
 ./export.sh \
@@ -59,19 +91,18 @@ output/pm4n/Flow_Controller_Panel-F_Silkscreen.preview.png
 output/pm4n/Flow_Controller_Panel-B_Silkscreen.pm4n
 output/pm4n/Flow_Controller_Panel-B_Silkscreen.preview.png
 ```
-#### 4.1. Check the layer preview
+#### 2.1. Check the layer preview
 Check the `output/pm4n/Flow_Controller_Panel-*.preview.png` images — traces should appear black on white background.
 - background = UV exposed = resist removed = etched away
 - traces = dark = resist kept = copper stays
 
-#### 4.2. Check the printer exposure
+#### 2.2. Check the printer exposure
 Open the `.pm4n` in `Chitubox Basic` slicer to visually verify before printing.
 
-#### 4.3. Adjust exposure
+#### 2.3. Adjust exposure
 Start with `--exposure 60` and bracket from there — Bungard presensitized at 405nm typically lands between 30–120s depending on board vintage and storage.
 
-
+### 3. Export single layer (e.g. copper)
-#### Export single layer (e.g. copper)
 
 Export the front layer as gerber
 ```bash

diagnose/decode_prev.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""
+Decode the PREV (preview thumbnail) section from a pm4n file and save as PNG.
+Run: python3 decode_prev.py <file.pm4n>
+"""
+import struct, sys
+from pathlib import Path
+
+path = sys.argv[1]
+data = open(path, 'rb').read()
+
+prev_off = data.find(b'PREV')
+print(f"PREV at 0x{prev_off:06X}")
+
+# PREV section layout (from ANYCUBIC format docs and common reverse-engineering):
+# +0x00  "PREV"
+# +0x04  u32  section_length
+# +0x08  u32  image_width
+# +0x0C  u32  image_height
+# +0x10  u32  image_data_length
+# +0x14  image data (RGB565 or RLE)
+
+for hdr_size in [0x14, 0x18, 0x10, 0x0C]:
+    w = struct.unpack_from('<I', data, prev_off + 0x08)[0]
+    h = struct.unpack_from('<I', data, prev_off + 0x0C)[0]
+    dlen = struct.unpack_from('<I', data, prev_off + 0x10)[0]
+    print(f"  PREV+0x08: w={w}, h={h}, dlen={dlen}")
+    if 100 < w < 2000 and 100 < h < 2000:
+        print(f"  -> plausible dimensions {w}x{h}, data at +0x14, {dlen} bytes")
+        break
+
+# Show first 64 bytes of PREV section
+print(f"\nPREV raw (first 64 bytes):")
+chunk = data[prev_off:prev_off+64]
+for i in range(0, 64, 16):
+    row = chunk[i:i+16]
+    hex_p = ' '.join(f'{b:02X}' for b in row)
+    u_vals = [struct.unpack_from('<I', row, j)[0] for j in range(0, min(16,len(row))-3, 4)]
+    print(f"  +0x{i:02X}  {hex_p:<48}  {u_vals}")
+
+# Try to decode as RGB565 (common for ANYCUBIC previews)
+# Each pixel = 2 bytes, little-endian RGB565
+# R = bits[15:11], G = bits[10:5], B = bits[4:0]
+img_data_off = prev_off + 0x14
+img_data_len = struct.unpack_from('<I', data, prev_off + 0x10)[0]
+w = struct.unpack_from('<I', data, prev_off + 0x08)[0]
+h = struct.unpack_from('<I', data, prev_off + 0x0C)[0]
+
+print(f"\nAttempting RGB565 decode: {w}x{h}, {img_data_len} bytes at 0x{img_data_off:06X}")
+if 0 < w < 2000 and 0 < h < 2000 and img_data_len == w * h * 2:
+    print(f"  Size matches RGB565 ({w}*{h}*2={w*h*2}) ✓")
+    try:
+        from PIL import Image
+        pixels = []
+        raw = data[img_data_off:img_data_off + img_data_len]
+        for i in range(0, len(raw)-1, 2):
+            px = raw[i] | (raw[i+1] << 8)
+            r = ((px >> 11) & 0x1F) << 3
+            g = ((px >> 5) & 0x3F) << 2
+            b = (px & 0x1F) << 3
+            pixels.append((r, g, b))
+        img = Image.new('RGB', (w, h))
+        img.putdata(pixels)
+        out = Path(path).with_suffix('.prev_thumb.png')
+        img.save(out)
+        print(f"  Saved thumbnail to {out}")
+    except Exception as e:
+        print(f"  Error: {e}")
+else:
+    print(f"  Size mismatch or bad dims, skipping decode")
+    print(f"  Expected {w*h*2} bytes for RGB565, got {img_data_len}")

diagnose/decode_rle_blocks.py

@@ -0,0 +1,76 @@
+#!/usr/bin/env python3
+"""
+Decode the composite block and first layer block from a pm4n file,
+report pixel counts and first/last pixel values.
+Run: python3 decode_rle_blocks.py output/pm4n/Flow_Controller_Panel-Front.pm4n
+"""
+import struct, sys
+
+path = sys.argv[1]
+data = open(path, 'rb').read()
+fsize = len(data)
+print(f"File: {path}  ({fsize} bytes)\n")
+
+LAYE_HDR = 0x20
+STRIDE   = 0x20
+
+laye = data.find(b'LAYE')
+n_entries    = struct.unpack_from('<I', data, laye + 0x10)[0]
+composite_off = struct.unpack_from('<I', data, laye + 0x14)[0]
+block_size   = struct.unpack_from('<I', data, laye + 0x18)[0]
+
+print(f"LAYE n_entries={n_entries}  composite_off=0x{composite_off:06X}  block_size={block_size}")
+
+def decode_rle(data, offset, max_bytes):
+    """Decode RLE starting at offset, stop after max_bytes read. Returns pixel list."""
+    pixels = []
+    i = offset
+    end = min(offset + max_bytes, len(data))
+    while i + 1 < end:
+        b0 = data[i]; b1 = data[i+1]
+        nibble = (b0 >> 4) & 0xF
+        colour = 255 if nibble == 0xF else 0
+        run = ((b0 & 0x0F) << 8) | b1
+        run += 1
+        pixels.extend([colour] * run)
+        i += 2
+    return pixels, i - offset
+
+def summarize_block(name, offset, size):
+    print(f"\n{name} at 0x{offset:06X}, {size} bytes:")
+    if offset + size > fsize:
+        print(f"  *** PAST EOF (file ends at 0x{fsize:06X}) ***")
+        size = max(0, fsize - offset)
+    
+    # Show first 16 bytes raw
+    raw = data[offset:offset+16]
+    print(f"  First 16 bytes: {raw.hex()}")
+    
+    # Decode RLE
+    pixels, bytes_consumed = decode_rle(data, offset, size)
+    print(f"  Decoded: {len(pixels)} pixels from {bytes_consumed} bytes")
+    print(f"  Expected: {9024*5120} pixels ({9024}×{5120})")
+    
+    if pixels:
+        whites = sum(1 for p in pixels if p >= 128)
+        blacks = len(pixels) - whites
+        print(f"  White px: {whites} ({100*whites/len(pixels):.1f}%)")
+        print(f"  Black px: {blacks} ({100*blacks/len(pixels):.1f}%)")
+        
+        # Check if it decodes to correct pixel count
+        if len(pixels) == 9024 * 5120:
+            print(f"  ✓ Correct pixel count")
+        else:
+            print(f"  *** WRONG pixel count (off by {len(pixels) - 9024*5120})")
+        
+        # Show first few runs
+        print(f"  First pixels: {pixels[:20]}")
+
+# Composite block
+summarize_block("Composite block (block 0)", composite_off, block_size)
+
+# First entry block  
+e0_base = laye + LAYE_HDR
+e0_data_off = struct.unpack_from('<I', data, e0_base + 0x14)[0]
+e0_data_sz  = struct.unpack_from('<I', data, e0_base + 0x18)[0]
+summarize_block("Entry[0] block", e0_data_off, e0_data_sz)

diagnose/diagnose_pm4n.py

@@ -60,3 +60,48 @@ for off in range(0, fsize - 3, 4):
         # Show context
         section_hint = next((f"sec[{i}]+{off-s:d}" for i,(s,l) in enumerate(sections) if s <= off < s+l), "outside")
         print(f"  0x{off:06X}  {v:.1f}  ({section_hint})")
+
+# ---------------------------------------------------------------------------
+# Extra: dump LAYE entries with CORRECT field layout
+# (verified from binary probe: HDR=0x20, exposure@+0x00, data_off@+0x14, data_size@+0x18)
+# ---------------------------------------------------------------------------
+def find_tag2(d, tag):
+    i = d.find(tag)
+    return i if i >= 0 else None
+
+LAYE_HDR_SIZE2 = 0x20
+ENTRY_STRIDE2  = 0x20
+
+laye2 = find_tag2(data, b'LAYE')
+if laye2 is not None:
+    n_entries2     = struct.unpack_from('<I', data, laye2 + 0x10)[0]
+    composite_off2 = struct.unpack_from('<I', data, laye2 + 0x14)[0]
+    block_size2    = struct.unpack_from('<I', data, laye2 + 0x18)[0]
+    print(f"\nLAYE (correct layout) at 0x{laye2:06X}:")
+    print(f"  n_entries    = {n_entries2}")
+    print(f"  composite_off= 0x{composite_off2:06X}")
+    print(f"  block_size   = {block_size2}")
+    expected_off = composite_off2
+    ok = True
+    for i in range(n_entries2):
+        base = laye2 + LAYE_HDR_SIZE2 + i * ENTRY_STRIDE2
+        if base + ENTRY_STRIDE2 > fsize:
+            break
+        exp     = struct.unpack_from('<f', data, base + 0x00)[0]
+        z_pos   = struct.unpack_from('<f', data, base + 0x04)[0]
+        d_off   = struct.unpack_from('<I', data, base + 0x14)[0]
+        d_sz    = struct.unpack_from('<I', data, base + 0x18)[0]
+        expected_off += block_size2
+        match = "OK" if d_off == expected_off and d_sz == block_size2 else "*** MISMATCH ***"
+        if match != "OK":
+            ok = False
+        print(f"  entry[{i:2d}]: exp={exp:.1f}s  z={z_pos:.2f}mm  "
+              f"data_off=0x{d_off:06X} (exp 0x{expected_off:06X})  "
+              f"size={d_sz}  {match}")
+    if ok:
+        print("  All entries consistent ✓")
+    # Also confirm global header px dimensions
+    gw = struct.unpack_from('<I', data, 0x7C)[0]
+    gh = struct.unpack_from('<I', data, 0x80)[0]
+    print(f"\nGlobal header px dimensions: {gw}×{gh}  "
+          f"({'✓ correct for Mono 4' if gw==9024 and gh==5120 else '*** WRONG ***'})")

diagnose/probe_laye.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+"""
+probe_laye.py — find where 9024 and 5120 are actually stored in a pm4n file,
+and dump LAYE + Mode sections in detail.
+Run: python3 probe_laye.py Dummy.pm4n
+"""
+import struct, sys
+
+path = sys.argv[1]
+data = open(path, 'rb').read()
+fsize = len(data)
+print(f"File: {path}  ({fsize} bytes)\n")
+
+# Search for 9024 and 5120 as u32 LE, u16 LE, and as floats
+targets = {
+    'u32 9024':  struct.pack('<I', 9024),
+    'u32 5120':  struct.pack('<I', 5120),
+    'u16 9024':  struct.pack('<H', 9024),
+    'u16 5120':  struct.pack('<H', 5120),
+    'f32 9024':  struct.pack('<f', 9024.0),
+    'f32 5120':  struct.pack('<f', 5120.0),
+    'f32 153.408': struct.pack('<f', 153.408),
+    'f32 87.040':  struct.pack('<f', 87.040),
+}
+for label, needle in targets.items():
+    pos = 0
+    while True:
+        idx = data.find(needle, pos)
+        if idx < 0:
+            break
+        print(f"  {label} found at 0x{idx:06X} ({idx})")
+        pos = idx + 1
+
+print()
+
+# Dump LAYE section
+laye_off = data.find(b'LAYE')
+if laye_off >= 0:
+    print(f"LAYE raw (first 160 bytes from 0x{laye_off:06X}):")
+    for i in range(0, 160, 16):
+        row = data[laye_off+i : laye_off+i+16]
+        if not row: break
+        hex_p = ' '.join(f'{b:02X}' for b in row)
+        interp = []
+        for j in range(0, len(row)-3, 4):
+            u = struct.unpack_from('<I', row, j)[0]
+            f = struct.unpack_from('<f', row, j)[0]
+            interp.append(f"0x{u:08X}/f={f:.5g}")
+        print(f"  +0x{i:02X}  {hex_p:<48}  {' | '.join(interp)}")
+    print()
+
+# Dump Mode section
+mode_off = data.find(b'Mode')
+if mode_off >= 0:
+    print(f"Mode raw (first 96 bytes from 0x{mode_off:06X}):")
+    for i in range(0, 96, 16):
+        row = data[mode_off+i : mode_off+i+16]
+        if not row: break
+        hex_p = ' '.join(f'{b:02X}' for b in row)
+        interp = []
+        for j in range(0, len(row)-3, 4):
+            u = struct.unpack_from('<I', row, j)[0]
+            f = struct.unpack_from('<f', row, j)[0]
+            interp.append(f"0x{u:08X}/f={f:.5g}")
+        print(f"  +0x{i:02X}  {hex_p:<48}  {' | '.join(interp)}")
+    print()

gerber_to_pm4n.py

@@ -36,10 +36,30 @@ NATIVE_DPMM = LCD_W_PX / LCD_W_MM   # 58.824 dpmm  (1 px ≈ 17.001 µm)
 # ---------------------------------------------------------------------------
 # pm4n format constants  (reverse-engineered from Dummy.pm4n)
 # ---------------------------------------------------------------------------
-LAYE_TAG     = b'LAYE'
+LAYE_TAG      = b'LAYE'
-MODE_TAG     = b'Mode'
+MODE_TAG      = b'Mode'
-ENTRY_STRIDE = 0x20   # 32 bytes per layer entry
+ENTRY_STRIDE  = 0x20   # 32 bytes per layer entry
-LAYE_HDR_SIZE = 0x1C  # bytes before first entry within LAYE section
+LAYE_HDR_SIZE = 0x20   # bytes before first entry within LAYE section
+#
+# LAYE header layout (verified from binary probe):
+#   +0x00  tag  "LAYE"
+#   +0x04  tag  "RDEF"
+#   +0x08  u32  0
+#   +0x0C  u32  0xC4
+#   +0x10  u32  n_entries
+#   +0x14  u32  composite_image_offset
+#   +0x18  u32  block_size  (RLE bytes per block)
+#   +0x1C  f32  lift_height_mm
+#
+# LAYE entry layout (0x20 bytes each, immediately after header):
+#   +0x00  f32  exposure_sec        <-- was wrongly assumed at +0x04
+#   +0x04  f32  z_position_mm
+#   +0x08  f32  layer_thickness_mm
+#   +0x0C  u32  unknown
+#   +0x10  u32  unknown
+#   +0x14  u32  image_data_offset   <-- was wrongly assumed at +0x18
+#   +0x18  u32  image_data_size     <-- was wrongly assumed at +0x1C
+#   +0x1C  f32  lift_speed
 
 
 def find_tag(data: bytes, tag: bytes, start: int = 0) -> int:
@@ -56,21 +76,8 @@ def find_tag(data: bytes, tag: bytes, start: int = 0) -> int:
 
 
 def count_laye_entries(data: bytes, laye_off: int) -> int:
-    """Count layer entries by scanning until EXTR tag or implausible float."""
+    """Read n_entries directly from LAYE header at +0x10."""
-    entry_start = laye_off + LAYE_HDR_SIZE
+    return unpack_u32(data, laye_off + 0x10)
-    n = 0
-    while True:
-        pos = entry_start + n * ENTRY_STRIDE
-        if pos + 4 > len(data):
-            break
-        word = data[pos:pos+4]
-        if word in (b'EXTR', b'MACH', b'Mode', b'HEAD', b'PREV', b'LAYE'):
-            break
-        v = struct.unpack_from('<f', data, pos)[0]
-        if not (0.0 < v < 1000.0):
-            break
-        n += 1
-    return n
 
 
 def unpack_u32(data: bytes, off: int) -> int:
@@ -183,10 +190,10 @@ def patch_pm4n(dummy_path: Path, image: Image.Image,
     log(f"  Image blocks: first=0x{composite_off:06X}, "
         f"old_size={old_block_size}, new_size={new_rle_size}")
 
-    # Patch exposure in all entries
+    # Patch exposure in all entries  (entry+0x00 = exposure_sec float)
     for i in range(n_entries):
         base = laye_off + LAYE_HDR_SIZE + i * ENTRY_STRIDE
-        struct.pack_into('<f', raw, base + 0x04, exposure_sec)
+        struct.pack_into('<f', raw, base + 0x00, exposure_sec)
     log(f"  Exposure patched to {exposure_sec}s in {n_entries} entries")
 
     # Update block size in LAYE header and Mode header
@@ -195,15 +202,21 @@ def patch_pm4n(dummy_path: Path, image: Image.Image,
     struct.pack_into('<I', raw, mode_off + 0x48, new_rle_size)
     log(f"  Mode at 0x{mode_off:06X}")
 
-    # Update image offsets and sizes in all entries
+    # Update image offsets and sizes in all entries.
+    # Layout: [composite block][layer-0 block][layer-1 block]...
+    # entry+0x14 = data offset, entry+0x18 = data size
     for i in range(n_entries):
         base = laye_off + LAYE_HDR_SIZE + i * ENTRY_STRIDE
-        struct.pack_into('<I', raw, base + 0x18, composite_off + (i + 1) * new_rle_size)
+        struct.pack_into('<I', raw, base + 0x14, composite_off + (i + 1) * new_rle_size)
-        struct.pack_into('<I', raw, base + 0x1C, new_rle_size)
+        struct.pack_into('<I', raw, base + 0x18, new_rle_size)
 
     # Splice new image data (composite block + one block per layer, all identical)
-    n_blocks      = n_entries + 1
+    # Layout: composite block at composite_off, then one block per entry.
-    old_end       = composite_off + n_blocks * old_block_size
+    # The file contains exactly n_entries blocks total (composite counts as block 0;
+    # the last entry's data_off is therefore one block past the last stored block,
+    # which Chitubox tolerates). We mirror the same layout.
+    n_blocks  = n_entries          # composite + (n_entries-1) layer blocks = n_entries total
+    old_end   = composite_off + n_blocks * old_block_size
     raw[composite_off:old_end] = new_rle * n_blocks
 
     output_path.write_bytes(raw)