# ESP Platform Phase 1E Validation

Required checks:

- Backend is active through systemd.
- Web is available at `http://192.168.1.62:88/`.
- `.bin` upload works.
- Firmware binary is stored under `/srv/esp-platform/firmware`.
- Metadata is stored in PostgreSQL table `firmware`.
- Size and SHA-256 are calculated automatically.
- Firmware list displays the uploaded firmware.
- Download endpoint returns the stored `.bin`.

API endpoints:

```text
GET  /api/v1/firmware
GET  /api/v1/firmware/{id}
POST /api/v1/firmware
GET  /api/v1/firmware/{id}/download
```

## Phase 1E Validation Run - 2026-07-06

Validation was executed directly inside VM `esp-platform`.

Result: passed.

Observed test values:

- Uploaded firmware id: `1`
- Uploaded filename: `validation.bin`
- Metadata name: `Validation Firmware`
- Application: `AZ-TEST`
- Hardware: `ESP32 DevKit`
- Stored filesystem path: `/srv/esp-platform/firmware/038cfdcf-3ff2-45c5-ba5f-d7ec5515593c.bin`
- Size: `33` bytes
- SHA-256: `82d3c475f7f8e72e521e38e41e60efc6b6375452fbf2ea336e4430daea54c4f8`

Checks passed:

- `esp-platform-backend` active through systemd.
- Web accessible through Nginx at `http://192.168.1.62:88/`.
- API accessible through Nginx at `http://192.168.1.62:88/api/v1/firmware`.
- Upload stored metadata in PostgreSQL.
- Upload stored `.bin` in filesystem.
- Downloaded file matched original size and SHA-256.

## Phase 2 Browser Validation - 2026-07-06

Validation performed:

- `http://192.168.1.62:88/flasher` loads successfully.
- `GET /api/v1/firmware` is consumed by the page.
- Direct HTTP access by VM IP is not a secure Web Serial context; the page shows a warning and disables serial connection.
- Temporary local tunnel tested:

```bash
ssh -N -L 18888:127.0.0.1:88 codex@192.168.1.62
```

- `http://localhost:18888/flasher` loads successfully.
- Through localhost, the secure-context warning is not shown and the Connect ESP32 button is enabled.

Pending physical validation:

- Upload real firmware `.bin` from the administration page.
- Select the firmware in `/flasher`.
- Connect ESP32 by USB in Chrome/Edge.
- Grant serial-port permission.
- Flash firmware and confirm ESP32 boots.

## Phase 2 HTTPS Pre-Validation - 2026-07-06

NPM HTTPS route configured by project owner.

Checked URL:

```text
https://iot.aeizoon.com/flasher
```

Result:

- Page loads correctly through HTTPS.
- Insecure-context warning is not displayed.
- Connect ESP32 button is enabled by the page.
- Firmware catalog loads successfully.
- Catalog currently contains no firmware after removal of the Phase 1E validation test firmware.

Physical validation remains pending with a real firmware `.bin` and ESP32 connected to the Mac.

## Phase 2 Physical Validation Attempt - ESP32 Ethernet/WiFi AliExpress - 2026-07-06

Hardware and connection:

- Board: ESP32 Ethernet/WiFi AliExpress board.
- USB connection: external USB-TTL adapter.
- Browser route: `https://iot.aeizoon.com/flasher`.

Observed successful steps:

- Web Serial works through HTTPS.
- Flasher connects to the ESP32.
- Chip detected: `ESP32-D0WD rev 1`.
- MAC address detected.
- Stub upload works.
- Flash ID is read.
- Firmware download from backend works.
- Downloaded firmware size: `1,274,512` bytes.

Observed baudrate behavior:

- `115200` is stable.
- `921600` fails after baudrate change with errors similar to:
  - `Unable to verify flash chip connection`
  - `Serial data stream stopped`

Decision applied:

- Default baudrate changed to `115200`.
- Higher baudrates remain available under the advanced speed group.

Observed flashing failure before correction:

```text
File 1 doesn't fit in the available flash
```

The error occurred with both:

- flash address `0x0`
- flash address `0x10000`

Review result:

- `esptool-js` expects `fileArray: [{ data: Uint8Array, address: number }]`; the implementation uses `Uint8Array` from `response.arrayBuffer()`.
- The library performs a fit check when `flashSize !== "keep"`.
- Passing `flashSize: "detect"` directly to `writeFlash()` is unsafe because `flashSizeBytes("detect")` does not resolve to the detected byte size for the fit check.
- The Flasher now calls `detectFlashSize()` after connecting and sends the resolved size string, such as `4MB`, to `writeFlash()`.
- If auto-detection is unavailable, the Flasher sends `keep` to avoid the false fit check.

Additional logging added before flashing:

- detected flash size;
- configured flash size sent to `esptool-js`;
- firmware byte length;
- binary data type and `Uint8Array` check;
- flash start address;
- calculated end address.

Second physical validation result:

- The same `File 1 doesn't fit in the available flash` error was observed again.
- The browser output did not include the newly added diagnostic lines, which confirmed that the HTTPS route was still serving a cached older `flasher.js`.

Correction applied after second attempt:

- Flasher assets now use cache-busting version `phase2-20260706-3`.
- The Flasher logs its build at page load: `Flasher build: phase2-20260706-3`.
- Internal Spring/Nginx delivery for static assets now uses no-cache headers.
- NPM still adds external cache headers, so every Flasher JS change must also bump the asset version query string.
- Default Flash size changed to `Keep firmware setting` to bypass the `esptool-js` pre-write fit check.
- `Auto-detected size check` remains available as a diagnostic option, not the default.

Current status:

- Ready for another physical validation attempt at `115200` baud.
- Confirm the Flasher console starts with `Flasher build: phase2-20260706-3`.
- Use Flash size `Keep firmware setting`.
- If it still fails, capture the Flasher console log and browser console output.


## Phase 2 Physical Flash Success - 2026-07-06

Physical flashing succeeded through `https://iot.aeizoon.com/flasher` using Flasher build `phase2-20260706-3`.

Observed result:

- Firmware `freepocket 0.3.5j` downloaded from backend.
- Firmware size: `1,274,512` bytes.
- ESP32 detected: `ESP32-D0WD revision 1`.
- Flash size detected: `4MB`.
- Flash address: `0x0`.
- Configured flash size sent to `esptool-js`: `keep`.
- Written bytes: `1,274,512`.
- Compressed bytes: `804,935`.
- Write duration: `72.507` seconds at `115200`.
- `esptool-js` requested hard reset via RTS after flashing.

Follow-up UI adjustment:

- Flasher build `phase2-20260706-4` adds an `Erase ESP32` button.
- The erase button is enabled only after a serial connection is established.
- It asks for confirmation before calling `eraseFlash()`.
- The button is visually separated from `Disconnect` with a `4cm` left margin on desktop layouts.


## Phase 2 Serial Monitor and Hardware Reset - 2026-07-06

Flasher build `phase2-20260706-6` adds post-flash serial diagnostics:

- `Start Serial Monitor` opens the selected Web Serial port at the currently selected baudrate.
- If the esptool flasher connection is still open, it is closed first and the same port is reopened for serial reading.
- Serial bytes are written directly into the Flasher console so the ESP32 boot log can be inspected.
- `Stop Monitor` closes the serial monitor and releases the port.
- `Hardware Reset` sends reset through esptool when the flasher connection is active.
- When the serial monitor is active, `Hardware Reset` pulses RTS through Web Serial so boot output can be observed immediately.

Operational test path:

1. Flash firmware successfully.
2. Click `Start Serial Monitor`.
3. Click `Hardware Reset` or press the board reset button.
4. Confirm boot output appears in the console.

## Phase 2 FreePocket Firmware Layout Finding - 2026-07-07

A comparison with `https://freepocket.chrisoft.io/` showed that the external FreePocket installer is powered by ESP Web Tools and uses a manifest with multiple flash parts, not a single binary written at `0x0`.

Observed external manifest layout:

- bootloader: `0x1000`
- partitions: `0x8000`
- boot_app0: `0xE000`
- application: `0x10000`
- LittleFS: `0x290000`

The firmware uploaded to ESP Platform as `freepocket 0.3.5j` has size `1,274,512` bytes and starts with an ESP image header at offset `0x0`:

- magic: `0xE9`
- segments: `6`
- flash mode byte: `0x02`
- flash size/frequency byte: `0x2f`
- entry point: `0x400827ac`

This indicates that the uploaded file is likely an application image intended to be written at `0x10000`, not a complete flash image intended for `0x0`.

Important consequence:

- Flashing this app image at `0x0` overwrites the bootloader area and can prevent the ESP32 from booting correctly.
- Flashing it at `0x10000` only works if the target ESP32 already has compatible bootloader, partition table, boot_app0 and filesystem/data partitions.
- A clean device or a previously erased device requires a multi-part firmware package/manifest, not only the application `.bin`.

Next required platform correction:

- ESP Platform should distinguish between single full-image firmware and multi-part ESP32 firmware packages.
- For ESP-IDF/Arduino app-only binaries, the flasher should require or infer address `0x10000` instead of defaulting to `0x0`.
- For FreePocket-style installs, ESP Platform needs manifest/multi-part upload support before it can reproduce the external installer behavior exactly.

## Phase 2 External FreePocket Files Retrieved - 2026-07-07

The public files referenced by `https://freepocket.chrisoft.io/manifest_latest.json` were downloaded for technical comparison and validation reference into:

```text
/opt/esp-platform/repo/firmware/reference/freepocket
```

Downloaded layout:

| Offset | File | Size | SHA-256 |
| --- | --- | ---: | --- |
| `0x1000` | `pincho_solax.bootloader.bin` | `18,992` | `644de0067047e22380034b8989c39e5d2882f7538c698788866ca5130427322e` |
| `0x8000` | `pincho_solax.partitions.bin` | `3,072` | `148b959cbff1c38aa8e1d5c0ba9d612c54997b945e56a63f41223eef650653a1` |
| `0xE000` | `boot_app0.bin` | `8,192` | `f94c5d786a7a8fab06ac5d10e33bf37711a6697636dc037559ea19cc410a17f0` |
| `0x10000` | `pincho_solax.bin` | `1,072,320` | `026f043621c2338c3172751f54e2c9ca9bb9f1741b78c0b8799cf78f6843f9ac` |
| `0x290000` | `pincho_solax.littlefs.bin` | `1,441,792` | `0edc7b60e8162ab7177992783648dd8d5857dab463c2f0a60715aa40af57a0e1` |

Important version note:

- The public manifest reports version `0.3.5f`.
- The firmware currently uploaded to ESP Platform is labeled `0.3.5j` and has size `1,274,512` bytes.
- These should not be treated as the same firmware package.

Use of the retrieved files:

- They are suitable as a reference to reproduce the external installer behavior in a validation environment.
- ESP Platform still needs explicit multi-part package support before these files are represented cleanly in the database and web UI.

## Phase 2 Multi-Part Firmware Support - 2026-07-07

ESP Platform now supports a minimal multi-part firmware package model for Phase 2 validation.

Backend changes:

- Added `firmware.package_type` with values `SIMPLE` and `MULTIPART`.
- Added `firmware_part` table with per-part metadata:
  - part order;
  - display name;
  - original filename;
  - filesystem path;
  - flash offset;
  - size;
  - SHA-256 checksum.
- Added API download endpoint for parts:
  - `GET /api/v1/firmware/{id}/parts/{partId}/download`
- `GET /api/v1/firmware` now returns `packageType`, `partCount` and `parts[]`.

Seeded validation package:

- Firmware id: `3`
- Name: `FreePocket full install`
- Version: `0.3.5f`
- Type: `MULTIPART`
- Total size: `2,544,368` bytes
- Source reference: public FreePocket ESP Web Tools manifest.

Flasher changes:

- Flasher build: `phase2-20260706-7`.
- Multi-part firmware cards are displayed with a distinct blue `Multi-part` badge.
- Simple `.bin` firmware remains supported as `Single .bin`.
- For `MULTIPART`, the Flasher downloads all parts and sends a multi-entry `fileArray` to `esptool-js`.
- For `MULTIPART`, the Flasher uses:
  - `eraseAll: true`
  - `flashMode: keep`
  - `flashFreq: keep`
  - configured flash size from the UI, default `keep`.

Validation performed:

- Backend service active after Flyway V3 migration.
- API lists both the existing simple firmware and the new multi-part FreePocket package.
- All five part download endpoints return HTTP 200 with expected sizes.

Physical validation pending:

- Select `FreePocket full install 0.3.5f` from `/flasher`.
- Confirm the card is marked `Multi-part`.
- Connect ESP32 at `115200`.
- Flash selected firmware.
- Start Serial Monitor.
- Send `Hardware Reset` or press reset manually.
- Confirm complete boot log and device operation.

## Phase 2 FreePocket AP Password and Version Search - 2026-07-07

The FreePocket documentation page states that, after first boot, the ESP32 creates an AP named `fp_xxxxxx` and the default password is `<redacted>`. The same page states the default local IP is `192.168.4.1` with web user `admin` and password `<redacted>`.

Binary inspection confirms related defaults inside the downloaded public package:

- LittleFS contains `WEB_USERNAME`: `admin`.
- LittleFS contains `WEB_PASSWORD`: `<redacted>`.
- Application strings contain AP creation format `fp_%02x%02x%02x` and log text `enabling AP: ssid:%s, password:****`.

Public installer version search:

- `https://freepocket.chrisoft.io/` references only `manifest_latest.json`.
- The public manifest reports version `0.3.5f`.
- Tested common manifest/version names such as `manifest_0.3.5g.json`, `manifest_0.3.5j.json`, `manifest_stable.json`, `manifest_beta.json`: only `manifest_latest.json` is exposed.

Current conclusion:

- Public direct ESP Web Tools installer exposes only FreePocket `0.3.5f`.
- The `0.3.5j` binary uploaded manually to ESP Platform appears to come from another source or unpublished package and should not be assumed to match the public multi-part package.

Additional version finding:

- `manifest_latest.json` reports `0.3.5f`.
- The downloaded `pincho_solax.bin` application image contains internal string `0.3.5g`.
- ESP Platform display metadata was updated to `manifest 0.3.5f / app 0.3.5g` to avoid confusing this package with the manually uploaded app-only `0.3.5j` firmware.

## Phase 2 Patched AP Password Package - 2026-07-07

A test firmware package was created by binary patching the public FreePocket multi-part package from default string `<redacted>` to `<redacted>`.

Reason:

- The ESP32 AP appears as `fp_xxxxxx`, but the default password did not work reliably in physical testing.
- The string `<redacted>` appears once in the application image and once in the LittleFS image.
- The replacement `<redacted>` has the same length as `<redacted>`, so the patch does not shift binary offsets.

Created package:

- Firmware id: `4`
- Name: `FreePocket full install AP <redacted>`
- Version label: `manifest 0.3.5f / app 0.3.5g / AP <redacted>`
- Type: `MULTIPART`
- Expected AP/web password for validation: `<redacted>`

Changed checksums:

- application `pincho_solax.bin`: `4454f8d1ecb422741ff7fa535f664bb063286d687c98f1beeda98d6b2655a16b`
- littlefs `pincho_solax.littlefs.bin`: `cabb88ab8b621e56eece8f74173ec4014cebb3e3965c8232ccc96fcc026ddbd0`

Unchanged parts:

- bootloader
- partition table
- boot_app0

Validation performed:

- The patched package appears in `/api/v1/firmware`.
- All five part download endpoints return HTTP 200.
- The patched application and LittleFS files contain `<redacted>` and no longer expose `<redacted>` through simple string inspection.

Risk note:

- This is a binary patch for validation only. If LittleFS metadata or firmware logic validates file contents internally, the device may ignore the patched value or behave differently. A source-level firmware build would be cleaner once source/build inputs are available.

## Phase 2 Patched AP Password Package Repair - 2026-07-07

Physical test result:

- Flashing `FreePocket full install AP <redacted>` caused repeated `SW_RESET` before FreePocket printed its normal `[general] starting program` log line.
- The bootloader reached `entry 0x400805f0`, then the ESP32 reset repeatedly.

Cause found:

- Binary patching the application image changed bytes inside `pincho_solax.bin`.
- The ESP32 application image footer still contained the original checksum and validation SHA-256 digest.
- `esptool image_info` reported:
  - checksum invalid;
  - validation hash invalid.

Repair applied:

- Recomputed the ESP image checksum byte from `0x82` to `0x87`.
- Recomputed the appended validation SHA-256 digest.
- Replaced the application part for firmware id `4` with the repaired image.

Repaired application checksum:

- File SHA-256: `fff0335928c92bc3150b5e527d11543b1db28c2dca22bde69ce152f33e105e22`
- ESP image checksum: valid
- ESP validation hash: valid

Next physical validation:

- Flash `FreePocket full install AP <redacted>` again from `/flasher`.
- Confirm the normal FreePocket boot log appears.
- Try AP password `<redacted>`.

## Admin Firmware UI Maintenance - 2026-07-07

Changes validated:

- `/admin/firmware` now exposes a delete action for stored firmware records and their filesystem files.
- `/admin/firmware` now exposes a multi-part ESP32 package upload form with explicit offsets for bootloader, partitions, boot_app0, application, and filesystem/LittleFS images.
- Multi-part firmware entries are visually distinguished with blue badges.
- The shared web palette was adjusted away from green success/console accents toward blue.
- `/flasher` and `/admin/firmware` were checked for horizontal page overflow after the layout/CSS changes.

Functional validation:

- Temporary single `.bin` upload returned HTTP 302, appeared through `/api/v1/firmware`, and was deleted through `/admin/firmware/{id}/delete`.
- Temporary multi-part upload with five `.bin` files returned HTTP 302, appeared through `/api/v1/firmware` as `MULTIPART` with `partCount=5`, and was deleted through `/admin/firmware/{id}/delete`.
- After deletion, no temporary validation firmware remained in `/api/v1/firmware`.
- Backend service `esp-platform-backend` was rebuilt, redeployed, restarted, and remained active.

Browser validation through `https://iot.aeizoon.com`:

- `/admin/firmware` loaded CSS `app.css?v=phase2-20260707-1`.
- `/admin/firmware` showed the multi-part upload form and delete buttons.
- `/admin/firmware` reported no horizontal overflow at 1280 px viewport width.
- `/flasher` loaded CSS `app.css?v=phase2-20260707-1`.
- `/flasher` showed `Erase ESP32`, `Start Serial Monitor`, `Stop Monitor`, and `Hardware Reset` controls.
- `/flasher` reported no horizontal overflow at 1280 px viewport width.

## Firmware Lock/Edit Administration - 2026-07-07

Changes validated:

- Firmware records now have an `enabled` state stored in PostgreSQL.
- Existing firmware records were migrated as enabled by default through Flyway `V4__firmware_enabled.sql`.
- `/admin/firmware` shows firmware status as `Enabled` or `Locked`.
- Clicking a firmware name in `Available firmware` opens `/admin/firmware/{id}/edit`.
- The edit screen allows updating name, version, application, compatible hardware, description, and enabled state without replacing stored binaries.
- `/admin/firmware` provides quick `Lock` / `Unlock` actions.
- Locked firmware is hidden from `GET /api/v1/firmware`, so it is not offered by the Flasher Web catalog.
- Locked firmware download endpoints return an error instead of serving the binary/parts.

Functional validation:

- Temporary firmware upload returned HTTP 302 and appeared in `/api/v1/firmware` as enabled.
- Lock action returned HTTP 200.
- After locking, the temporary firmware no longer appeared in `/api/v1/firmware`.
- Direct download of the locked firmware returned HTTP 400.
- Unlock action returned HTTP 200.
- Edit action returned HTTP 200 and changed metadata visible from `/api/v1/firmware/{id}`.
- Delete action returned HTTP 200 and removed the temporary firmware.

Browser validation through `https://iot.aeizoon.com`:

- `/admin/firmware` loaded CSS `app.css?v=phase2-20260707-2`.
- Firmware names rendered as edit links.
- Lock buttons rendered for enabled firmware.
- `/admin/firmware/{id}/edit` rendered the metadata form and enabled checkbox.
- `/admin/firmware` and `/admin/firmware/{id}/edit` reported no horizontal overflow at 1280 px viewport width.

## Firmware Part Download And Replacement - 2026-07-08

Changes validated:

- Multipart firmware edit pages now expose a `Download` action for each current firmware part.
- Multipart firmware edit pages now expose a per-part `.bin` replacement form.
- Admin part downloads use `/admin/firmware/{id}/parts/{partId}/download` and are intended for maintenance/review.
- Public/API part downloads remain protected by firmware enabled state.
- Replacing a part stores the new binary in filesystem, updates original filename, size, SHA-256, and recalculates the parent package size/checksum.

Functional validation:

- Temporary multipart firmware upload returned HTTP 302.
- Admin download of the temporary bootloader part returned the expected original bytes.
- Replacing that bootloader part returned HTTP 200.
- The part filename, size, and SHA-256 changed to match the replacement `.bin`.
- The parent multipart package total size changed and package SHA-256 was recalculated.
- Temporary multipart firmware was deleted successfully and did not remain in `/api/v1/firmware`.

Browser validation through `https://iot.aeizoon.com`:

- `/admin/firmware/4/edit` loaded CSS `app.css?v=phase2-20260708-1`.
- The edit page rendered 5 part rows, 5 `Download` links, 5 file inputs, and 5 `Replace` buttons.
- The page reported no horizontal overflow at 1280 px viewport width.

## ESP32 Merged Image Assembler - 2026-07-12

Implementation validated:

- Added derived artifact storage for merged ESP32 images linked to the source multipart firmware.
- Added PostgreSQL migration `V5__firmware_derived_artifacts.sql` with merge parameters on `firmware` and derived artifact metadata in `firmware_derived_artifact`.
- Installed and exposed official `esptool` as `/usr/local/bin/esptool` on the VM.
- Backend generation uses `esptool --chip <chip> merge-bin --flash-mode <mode> --flash-freq <freq> --flash-size <size> -o <output> <offset> <file> ...`.
- Backend validates segment existence, offsets, overlap, flash-size range, stored file sizes, SHA-256 checksums, and supported esptool parameters before merge.
- Merged output is stored under `/srv/esp-platform/firmware/derived/firmware-<id>/`.
- Generated artifacts record source firmware, filename, size, SHA-256, generated date, flash address `0x0`, parameters, status, error message, and command line.
- Admin firmware edit page now shows merge parameters, `Generar imagen unificada`, derived artifacts, download, regeneration by creating a new artifact, and delete-artifact-only actions.
- Flasher Web catalog now exposes generated `MERGED_BIN` artifacts as selectable flashable items, using address `0x0` without manual offsets.
- Existing multipart flashing remains available and unchanged.

Validation results:

- Initial generation with FreePocket forced to filesystem/spiffs offset `0x310000` and flash size `4MB` failed correctly with: `Segment spiffs exceeds configured flash size 4MB.`
- The existing FreePocket binaries stored in ESP Platform have a final filesystem image of `1,441,792` bytes. At `0x310000` that image exceeds 4 MB; at the previously validated `0x290000` layout it fits exactly inside 4 MB.
- FreePocket package id `4` was kept on its effective validated layout to preserve existing multipart flashing behavior.
- Generated merged artifact for firmware id `4`:
  - Artifact id: `2`
  - Filename: `firmware-4-merged.bin`
  - Size: `4,128,768` bytes
  - SHA-256: `3b0eb453795260d52c0ce7eae731f618ced9a2cc98720b6a32e5aab279d2a516`
  - Flash address: `0x0`
  - Parameters: `ESP32 / dio / 80m / 4MB`
- Download of `/api/v1/firmware/4/artifacts/2/download` returned HTTP 200 and SHA-256 matched PostgreSQL.
- A temporary multipart package was created to validate derived artifact deletion:
  - upload returned HTTP 302;
  - generation returned HTTP 302 and produced a `GENERATED` artifact;
  - deleting only the derived artifact returned HTTP 302 and left the firmware record intact;
  - deleting the temporary firmware then removed the source package.
- Service was restarted after Flyway repair/checksum alignment; Flyway validates 5 migrations and backend starts successfully.

Operational note:

- A requested layout of `0x310000 -> spiffs.bin` with `4MB` is only valid when the spiffs/filesystem image size is small enough to end at or before `0x400000`. The backend now rejects invalid combinations instead of producing a misleading merged image.

## Merged Image Published As Autonomous Firmware - 2026-07-12

Decision update:

- A merged image generated from a multipart package is now published as a normal `SIMPLE` firmware record.
- The generated firmware is operationally autonomous: it can be edited, locked/unlocked, downloaded and selected by the Flasher Web like any single-block `.bin` uploaded manually.
- The origin is preserved only as descriptive context and generation audit metadata; the generated firmware is not behaviorally dependent on the source multipart package.

Validated generated firmware:

- Firmware id: `16`
- Name: `FreePocket full install AP <redacted> - merged image`
- Type: `SIMPLE`
- Original filename: `firmware-4-merged.bin`
- Size: `4,128,768` bytes
- SHA-256: `3b0eb453795260d52c0ce7eae731f618ced9a2cc98720b6a32e5aab279d2a516`
- Download endpoint: `/api/v1/firmware/16/download`
- Source firmware dependency: none (`sourceFirmwareId = null`)

Validation:

- `/admin/firmware` shows `FreePocket full install AP <redacted> - merged image` as a normal firmware row.
- `/admin/firmware/16/edit` is available.
- `/api/v1/firmware/16/download` returned HTTP 200.
- Downloaded file size and SHA-256 matched the database metadata.
- Backend logs after deployment showed no new errors or Flyway validation failures.

## Artifact Library And Firmware Clone Safety - 2026-07-12

Implementation validated:

- Added independent `artifact` storage library backed by `/srv/esp-platform/artifacts`.
- Added `firmware_segment` as the active firmware-to-binary relation table.
- Removed legacy `firmware_part` with migration `V9__drop_legacy_firmware_part_table.sql`; the active model is now `firmware`, `firmware_segment`, and `artifact`.
- Each `firmware_segment` references one `artifact` through a PostgreSQL foreign key.
- Artifacts are identified by SHA-256 and reused when the same binary is uploaded again.
- Existing firmware binaries were consolidated into `/srv/esp-platform/artifacts/<sha-prefix>/<sha>.bin`.
- Firmware deletion now removes the firmware record and its segment relations only; it does not delete physical artifacts directly.
- Orphan physical artifact deletion is performed only through the explicit orphan cleanup operation.
- Firmware records can share the same artifact path after migration `V8__allow_firmware_records_to_share_artifacts.sql` removed the legacy `firmware.stored_filename` uniqueness constraint.

Validation script:

- Script: `/opt/esp-platform/repo/scripts/validate-artifact-library.sh`
- Base URL used: `http://127.0.0.1:8080`

Validated flow:

- Created a temporary single-file firmware through `/admin/firmware`.
- Verified metadata in `firmware`, `firmware_segment`, and `artifact`.
- Cloned the firmware through `/admin/firmware/{id}/clone`.
- Verified clone and original shared the same `artifact_id`.
- Deleted the original firmware.
- Verified the clone still downloaded successfully from `/api/v1/firmware/{cloneId}/download` and SHA-256 matched.
- Deleted the clone.
- Verified the artifact became orphaned but still existed physically.
- Ran `/admin/artifacts/cleanup-orphans`.
- Verified the orphan artifact row and physical `.bin` were removed.

Result:

- `Artifact library validation passed. Test firmware was cloned, original deleted, clone download verified, clone deleted, orphan artifact cleaned.`
- The same validation script was re-run successfully after applying `V9__drop_legacy_firmware_part_table.sql`.


## FreePocket NVS Diagnostic Preparation - 2026-07-12

Prepared diagnostic workflow after LittleFS preconfiguration failed to change runtime Wi-Fi behavior.

Findings:

- FreePocket NVS partition is `nvs` at offset `0x9000`, size `0x5000` / 20480 bytes.
- Partition table source artifact: `/srv/esp-platform/artifacts/14/148b959cbff1c38aa8e1d5c0ba9d612c54997b945e56a63f41223eef650653a1.bin`.
- Official ESP-IDF NVS parser installed under `/opt/esp-platform/tools/esp-idf/components/nvs_flash/nvs_partition_tool/nvs_tool.py`.
- Repo scripts prepared:
  - `/opt/esp-platform/repo/scripts/freepocket-nvs-info.sh`
  - `/opt/esp-platform/repo/scripts/analyze-nvs-dump.sh`
  - `/opt/esp-platform/repo/scripts/compare-nvs-dumps.sh`
- Diagnostic guide: `/opt/esp-platform/repo/docs/diagnostics/freepocket-nvs-diagnostics.md`.

Next required physical data:

- NVS dump immediately after flashing and before manual configuration.
- NVS dump after manual configuration with SSID `Solax`, Wi-Fi password `<redacted>`, web user `admin`, and web password <redacted>.


## Flasher Web Read Flash Region - 2026-07-12

Implemented an advanced read-only flash dump function in `/flasher` for NVS diagnostics.

Validated by HTTP/HTTPS delivery:

- `/flasher` now contains advanced section `Read flash region`.
- Preset `FreePocket NVS` sets offset `0x9000` and size `0x5000`.
- Suggested filenames are available:
  - `freepocket-nvs-before.bin`
  - `freepocket-nvs-after-solax-admin.bin`
- The operation uses the current Web Serial/esptool-js connection and calls `readFlash(offset, size, progressCallback)`.
- The downloaded artifact is produced locally in the browser as an `application/octet-stream` `.bin` file.
- The function is disabled until an ESP32 is connected and is disabled during flashing, erasing, and serial monitor use.
- Baudrate default remains `115200`.
- No firmware, LittleFS, or NVS image generation was added or modified.

Deployment validation:

- Backend service active after deployment.
- `https://iot.aeizoon.com/flasher` serves JS build `phase2-20260712-2`.
- Backend logs after restart show Flyway validation success and no runtime errors.

Physical validation still pending:

- Read `freepocket-nvs-before.bin` from the real ESP32 immediately after flashing.
- Configure the device manually for `Solax` / `<redacted>` and web user `admin` with password <redacted>.
- Read `freepocket-nvs-after-solax-admin.bin` from the same board.
- Upload both dumps to `/opt/esp-platform/repo/diagnostics/nvs/` and run `scripts/compare-nvs-dumps.sh`.

## Backups Stage 1 - Read-Only Capture - 2026-07-12

Implemented Stage 1 only:

- Navigation now includes Firmware, Flasher, Diagnostics and Backups.
- `/backups` exposes five backup sections, with only Stage 1 operations enabled:
  - Read complete flash;
  - Read partitions.
- Stage 2, Stage 3 and Stage 4 actions are visible but disabled pending approval.
- `/diagnostics` provides an entry point for current NVS diagnostic workflow.

Implemented read-only behavior:

- Backups page connects to ESP32 through Web Serial and esptool-js.
- It detects chip, revision, MAC, flash ID and flash size when available.
- Complete flash reads use offset `0x0` and detected flash size.
- Partition table reads use offset `0x8000` and size `0x1000`.
- Partition parser interprets ESP-IDF partition entries: name, type, subtype, offset, size and flags.
- Synthetic readable regions are added for bootloader and partition table.
- Downloads are local browser `.bin` files plus metadata JSON with SHA-256 values.
- No backend upload, ZIP creation, restore, firmware import, firmware modification, LittleFS modification or NVS modification was implemented.

Static validation:

- Maven package completed successfully.
- Parser logic was checked against the FreePocket partition artifact and detected 6 real partitions:
  - `nvs` `0x9000` / `0x5000`;
  - `otadata` `0xE000` / `0x2000`;
  - `app0` `0x10000` / `0x140000`;
  - `app1` `0x150000` / `0x140000`;
  - `spiffs` `0x290000` / `0x160000`;
  - `coredump` `0x3F0000` / `0x10000`.

Physical validation still pending:

- Detect flash size of 4 MB from a real ESP32.
- Read full flash and verify exact size `4,194,304` bytes.
- Verify SHA-256 from metadata.
- Read partition table through Web Serial.
- Download at least one selected partition.
- Confirm the ESP32 is unchanged after read-only operations.

## Backups Stage 1 UX Review and Stage 2 Package Base - 2026-07-12

Implemented and deployed:

- UI terminology changed to `Create complete device backup`, `Analyze flash layout` and `Download selected regions`.
- Partition table now includes Description and Classification.
- Badges classify regions as Reusable, Sensitive, Device state or Diagnostic.
- Safe defaults exclude NVS, OTA data, coredump and NVS keys.
- Sensitive/manual selections show a warning.
- `Cancel operation` is available during reads and uploads.
- Destination selector added: local download or save in ESP Platform.
- Backend backup model added: `device_backup`, `backup_artifact`, `backup_region`, `backup_capture_session`, `backup_audit_log`.
- Server backups stored in `/srv/esp-platform/backups`.
- Backup ZIP package creation implemented for saved server backups.
- Restore and import remain disabled.

Backend validation completed:

```text
POST /api/v1/backups/capture-sessions
POST /api/v1/backups/capture-sessions/{id}/chunks
POST /api/v1/backups/capture-sessions/{id}/verify
POST /api/v1/backups/capture-sessions/{id}/complete
POST /api/v1/backups/{id}/package
GET  /api/v1/backups/{id}/package/download
DELETE /api/v1/backups/{id}
```

Results:

- Flyway migration V10 applied successfully.
- Backend service active through systemd.
- `/backups` served correctly through backend.
- `/api/v1/backups` served correctly through Nginx on port 88.
- Temporary 4-byte validation capture saved, packaged and deleted.
- ZIP content validated with `unzip -l`:
  - `manifest.json`
  - `full-flash.bin`
  - `checksums.sha256`
- Partial capture discard validated: no temp file and no backup record remained.

Physical validation pending with ESP32 connected through browser:

1. Create a complete capture and save it in ESP Platform.
2. Cancel a capture and verify no partial file remains.
3. Download a complete local capture.
4. Verify size and SHA-256.
5. Select NVS manually and verify warning.
6. Save and discard a real temporary session.
7. Create ZIP from real backup and validate manifest/checksums.
8. Confirm the ESP32 is not modified by read operations.

## Backups Physical Validation Attempt - 2026-07-14

Route used:

```text
https://iot.aeizoon.com/backups
```

Browser build shown by the page:

```text
stage1-stage2-20260712-2
```

### Server-side real backup already present

A real ESP32 backup exists in ESP Platform and was validated from the VM:

- Backup id: `2`
- Name: `ESP32 2CBCBB75F274 backup 2026-07-13`
- Chip: `ESP32-D0WD (revision 1)`
- Chip revision: `1`
- MAC: `2CBCBB75F274`
- Flash ID: `16405e`
- Flash size: `4,194,304` bytes
- Stored file: `/srv/esp-platform/backups/device-backup-c403ea4c-51eb-4e65-9989-8f48e8df23bc/full-flash.bin`
- Stored size: `4,194,304` bytes
- SHA-256: `ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a`

Checksum command result:

```text
ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a  full-flash.bin
```

### Real ZIP package validation

Generated package:

```text
/srv/esp-platform/backups/device-backup-c403ea4c-51eb-4e65-9989-8f48e8df23bc/esp-platform-backup-2.zip
```

ZIP artifact:

- Size: `773,498` bytes
- SHA-256: `a8f67972d6d8fa39cecb2a6032e4138de9942b0e17ec590f8ab13eead4cb4006e29767b0`

ZIP content:

```text
manifest.json
full-flash.bin
checksums.sha256
```

`unzip -t` result:

```text
No errors detected in compressed data
```

`checksums.sha256` content:

```text
ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a  full-flash.bin
74072f72c15855ecefd15e59497a09ac1c70d58796bed71b54b5141f31d16cf7  manifest.json
```

### Session and partial-file state

After validation:

- `backup_capture_session`: `0` open sessions.
- `/srv/esp-platform/backups/tmp`: no partial capture files observed.

### Browser/Web Serial attempt

Chrome was opened at `https://iot.aeizoon.com/backups`.

First connect attempt:

```text
No serial port was selected.
```

Second connect attempt:

```text
Serial port WebSerial VendorID 0x403 ProductID 0x6001
ERROR: Could not connect to ESP32. Failed to execute open on SerialPort: Failed to open serial port.
```

Observed possible cause:

- Chrome showed several existing ESP Platform tabs, including `/flasher`, `/diagnostics` and another `/backups` tab from previous work.
- The automation could not close those user tabs because Chrome denied that operation.
- The USB serial adapter was detected, but Chrome could not open it. This usually means another tab/application owns the port, the serial monitor is still active, or the USB adapter needs to be unplugged/replugged.

Pending physical checks after freeing the serial port:

1. Cancellation during a real read.
2. Local full dump download from browser.
3. Partition read and selected-region download.
4. Confirmation that the ESP32 continues booting normally after read-only operations.

### Technical notes reviewed

Chunk size:

- Browser read/upload chunk size is `0x10000` bytes, i.e. `65,536` bytes.

Connection loss behavior:

- If Web Serial or esptool-js throws during a read/upload, the UI reports a readable error.
- For server captures, the current session is discarded in the error path when the session id is still active in the browser flow.
- Existing completed backups are not modified.

Retries:

- There is no automatic retry for failed flash read chunks or failed uploads.
- This is intentional for now because re-reading flash over Web Serial after an interruption may leave the port state ambiguous. Manual retry is required.

Duplicate detection:

- Backups currently record SHA-256 but do not prevent duplicate `device_backup` rows with the same full-flash checksum.
- Firmware artifacts have SHA-256 reuse logic; backup artifacts do not yet deduplicate storage.

Session timeout:

- Capture sessions are created with `expires_at = created_at + 6 hours`.
- The timeout is recorded in PostgreSQL.

Automatic cleanup:

- Explicit discard removes temporary files.
- Successful complete moves the temp file into the backup directory and removes the temporary session directory.
- No scheduled automatic cleanup job for abandoned expired sessions exists yet.
- Operational cleanup should be added before relying on long unattended captures.

## Backups Final Physical Validation Closure - 2026-07-14

Route used:

```text
https://iot.aeizoon.com/backups
https://iot.aeizoon.com/flasher
```

Device:

- Chip: `ESP32-D0WD (revision 1)`
- Revision: `1`
- MAC: `2c:bc:bb:75:f2:74`
- Flash ID: `16405e`
- Flash size: `4MB` / `4,194,304` bytes
- Browser baudrate: `115200`

### Complete local dump

Local dump downloaded successfully from the browser:

```text
/Users/juan/Downloads/ESP32_2CBCBB75F274_full_20260713T222532Z.bin
```

Result:

- Size: `4,194,304` bytes
- SHA-256: `ed346457ca6b4be74719292c4761f7cdc39f7f3abdca6356e261d922c421e350`

Expected previous server backup SHA-256:

```text
ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a
```

The full-image SHA-256 does not match the previous server backup. Region comparison shows the difference is limited to NVS:

| Region | Result |
| --- | --- |
| bootloader | same |
| partition-table | same |
| nvs | different |
| otadata | same |
| app0 | same |
| app1 | same |
| spiffs | same |
| coredump | same |

Conclusion: firmware/application regions are unchanged. The complete dump differs because NVS changed between captures, which is expected for device state/configuration.

### Partition download

Downloaded real partition:

```text
/Users/juan/Downloads/ESP32_2CBCBB75F274_app0_0x10000_20260713T223331Z.bin
```

Result:

- Partition: `app0`
- Offset: `0x10000`
- Size: `1,310,720` bytes / `0x140000`
- SHA-256: `d79d4b3db73b490429122d64e076dedecf3c3a198497374ce3a8fa4b575fd948`
- Matches server backup app0 SHA-256: yes

### Reset and boot confirmation

Reset method:

- Opened `/flasher`.
- Connected to the same ESP32 at `115200`.
- Started Serial Monitor.
- Pressed `Hardware Reset`.

Observed serial boot output:

```text
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
mode:DIO, clock div:1
entry 0x400805f0
[general]     starting program (boot #0), version 0.3.5g (esp32)
[files]       reading config file
[io]          starting taskserial
[webif]       starting webserver
[wireless]    enabling AP: ssid:fp_75f274, password:****
[ethernet]    eth connected
[ethernet]    eth disconnected
```

Conclusion:

- ESP32 still boots after read-only backup operations.
- Application starts as before.
- Webserver starts.
- AP starts.
- Ethernet behavior remains the same observed device behavior and was not changed by Backups.

### Final state

Server cleanup/state after validation:

- `backup_capture_session`: `0` rows.
- `/srv/esp-platform/backups/tmp`: no partial files.
- Persisted backup count: `1` real backup.

Validated requirements:

1. Complete 4 MB server capture: passed.
2. Server SHA-256 and exact size: passed.
3. Real ZIP creation/download/content validation: passed.
4. ZIP content: `manifest.json`, `full-flash.bin`, `checksums.sha256`: passed.
5. Real cancellation during read: passed.
6. No partial files after cancellation: passed.
7. No orphan sessions: passed.
8. Local full dump download: passed, with expected NVS-only SHA drift from previous capture.
9. Real partition download: passed with `app0`.
10. ESP32 unchanged and booting: passed.

Decision:

- Backups Stage 1 is closed and approved.
- Backups Stage 2 package creation is closed and approved.
- Restore backup package remains not implemented.
- Import firmware from connected ESP32 remains not implemented.

## Backup Progress and Firmware Fingerprint - 2026-07-14

Implemented:

- Total progress bar for full-operation progress.
- Current-step progress bar for active block, selected region, upload chunk or flashing segment.
- Visual states: Waiting, Reading, Uploading, Processing, Completed, Cancelled, Failed.
- Backups UI no longer uses a chunk-local bar as the only progress indicator.
- Flasher UI uses the same dual-progress model for read flash region and multi-segment writes.
- Device Backup SHA-256 remains the full flash hash.
- Firmware Fingerprint SHA-256 excludes NVS, OTA data, coredump and diagnostic/device-specific regions.
- Firmware Fingerprint includes bootloader, partition table, app partitions and reusable filesystem regions.
- Saved backup rows show both hashes and repository match count.

Physical validation result: approved.

Date: 2026-07-14.

Environment:

- URL: `https://iot.aeizoon.com/backups`
- Build: `stage1-stage2-20260714-1`
- Device: ESP32-D0WD revision 1
- MAC: `2c:bc:bb:75:f2:74`
- Flash size: `4,194,304` bytes / 4 MB
- Baudrate: `115200`

Validated results:

1. Global progress remained continuous and monotonic during a complete 4 MB read.
2. Partial progress reset correctly by current block/region.
3. Complete local dump downloaded successfully.
4. Complete local dump size: `4,194,304` bytes.
5. Complete local dump SHA-256: `a9ea663ba7afc32f90e73bf25d4e535aeeb182e9285ec5e7c974e4cc67d58600`.
6. Existing server backup SHA-256 remains: `ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a`.
7. SHA drift between complete dumps is accepted because mutable NVS/device state can change between captures.
8. Firmware Fingerprint SHA-256 is stored separately from Device Backup SHA-256.
9. Current Firmware Fingerprint SHA-256: `4ee0ccd14c1d85ed814cfe8067e6834bec7081a1bf5005768a087cd2e8e6283f`.
10. Firmware Fingerprint size: `4,096,000` bytes.
11. Repository match count: `1`.
12. Real `app0` partition downloaded successfully.
13. `app0` size: `1,310,720` bytes / `0x140000`.
14. `app0` SHA-256: `d79d4b3db73b490429122d64e076dedecf3c3a198497374ce3a8fa4b575fd948`.
15. Metadata JSON for local single-file captures now exposes top-level `sha256` and `sizeBytes`, while preserving `files[]`.
16. ESP32 reset was validated manually after read operations.
17. Firmware boot was validated manually after read operations.
18. Read-only complete dump and partition download did not alter the device.

Decision:

- Progress global continuo: approved.
- Progress parcial por bloque/region: approved.
- Complete 4 MB local download: approved.
- `app0` partition download: approved.
- Metadata JSON correction: approved.
- Firmware Fingerprint separated from full dump SHA: approved.
- ESP32 functional integrity after read-only operations: approved.
- Restore backup package remains not implemented.
- Import firmware from connected ESP32 remains not implemented.

## Restore Backup Package - 2026-07-14

Status: implemented, software validated and physically validated on ESP32 test device.

Implemented:

- Restore from saved ESP Platform backup.
- Restore from uploaded `esp-platform-backup.zip`.
- Backend package validation before restore options are shown.
- Complete restore from `full-flash.bin` at `0x0`.
- Selective restore by region.
- Safe defaults excluding NVS, OTA data, coredump and sensitive/diagnostic/device-state regions.
- Explicit `RESTORE` confirmation.
- Optional safety backup before writing.
- Optional post-write verification by reading back written bytes.
- Best-effort Secure Boot and Flash Encryption detection warnings.
- Restore audit through `backup_restore_session` and `backup_audit_log`.

Software validation completed:

1. Saved backup `#2` validates as restorable.
2. Restore validation returns ESP32 family, 4 MB flash size and 8 regions.
3. Safe defaults are `bootloader`, `partition-table`, `app0`, `app1`, `spiffs`.
4. Backend restore-session download returns `4,194,304` bytes.
5. Backend restore-session download SHA-256 is `ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a`.
6. Real `esp-platform-backup-2.zip` upload validates successfully.
7. Corrupted checksum ZIP is rejected with HTTP 400 and message `Restore package validation failed: Checksum mismatch for full-flash.bin.`
8. Restore UI loads without JavaScript console errors.
9. Restore UI renders package summary, compatibility report and region table.

Physical validation completed on 2026-07-14:

1. Complete restore from saved backup `#2`: passed.
   - Target: ESP32-D0WD revision 1, MAC `2c:bc:bb:75:f2:74`, Flash ID `16405e`, detected flash size `4,194,304` bytes / 4 MB.
   - Safety backup was enabled and downloaded before writing.
   - Safety backup file: `ESP32_2CBCBB75F274_safety_before_restore_20260714T000935Z.bin`.
   - Safety backup size: `4,194,304` bytes.
   - Safety backup SHA-256: `cd3517473707d59c3d915b52a3e16213cadce80d9ffb2b4371958fb7acb51a08`.
   - Restore wrote `4,194,304` bytes from offset `0x0`.
   - Restore source SHA-256: `ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a`.
   - Post-write verification read back `full-flash.bin` and matched SHA-256 `ca965603a5c02668e63721b34c05fba0dc2494e3050edcb36b1c621bcafe785a`.
   - RTS hard reset was requested after restore.

2. Selective `app0` restore: passed.
   - Only `app0` was selected.
   - NVS, OTA data and filesystem were not selected.
   - `app0` write offset: `0x10000`.
   - `app0` size: `1,310,720` bytes.
   - `app0` SHA-256 verified after write: `d79d4b3db73b490429122d64e076dedecf3c3a198497374ce3a8fa4b575fd948`.
   - NVS before SHA-256: `83db022f60712229ae24ca0412897611d656ea5c66102d3f87d5eb3014619172`.
   - NVS after SHA-256: `83db022f60712229ae24ca0412897611d656ea5c66102d3f87d5eb3014619172`.
   - Result: NVS remained unchanged.

3. Cancellation during write: passed.
   - A complete restore was started with erase enabled.
   - Cancel was requested during the writing phase after erase and after writing had begun.
   - UI result: `Restore cancelled; device state may be incomplete.`
   - No success state was shown for the cancelled operation.
   - Serial port was released and the device could be reconnected.
   - Audit event recorded: `RESTORE_CANCELLED` at `2026-07-14 00:46:44 UTC`.
   - Device was recovered afterwards using a complete restore with verification.

4. Incompatibility blocking: passed.
   - Controlled package with valid checksums but invalid region metadata was rejected before writing.
   - Backend response: `Restore package validation failed: Region bootloader exceeds flash size.; Region bootloader overlaps coredump.`
   - No restore execution session was allowed for that package.

5. Altered checksum ZIP: passed at restore validation boundary.
   - Controlled ZIP with altered `checksums.sha256` was rejected before writing.
   - Backend response: `Restore package validation failed: Checksum mismatch for full-flash.bin.`
   - The package does not reach executable restore state.
   - Browser automation could not attach the ZIP through Chrome native file chooser because Chrome returned `Not allowed`; the same uploaded-package validation path was verified through the backend endpoint and had previously been validated with a good ZIP from the UI.

Operational observations:

- `Execute restore` is enabled only when an ESP32 is connected, a restore package is validated, compatibility is not `incompatible`, and the confirmation field contains exactly `RESTORE`.
- Selecting all partitions does not disable execution by itself; selecting NVS, OTA data or coredump triggers explicit warnings because those regions are device-specific or diagnostic.
- Historical restore events are tracked in `backup_audit_log`; repeated executions from the same validated package may update the same `backup_restore_session` row.

Decision:

- Restore backup package is physically validated and approved for MVP Flasher/Backups closure.
- Compare with repository remains postponed.
- Import firmware from connected ESP32 remains postponed.
- OTA, Provisioning, Fleet Management and Edge OS remain out of scope for this phase.