mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-06-11 08:51:43 +00:00
Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a6413fb665 | |||
| cdcaa476f2 | |||
| a94c24c550 | |||
| a1f95fee58 | |||
| 24d76f8373 | |||
| 8e18351c73 | |||
| a827fd3b43 | |||
| 467a307a8d | |||
| 077fca9038 | |||
| b326e3f1a6 | |||
| 54cbc648e0 | |||
| aba4270ceb | |||
| 57b0188158 | |||
| f374a4a775 | |||
| 6d31cb2ad6 | |||
| 1619f4857e | |||
| 58d19ec303 |
@@ -0,0 +1,61 @@
|
||||
---
|
||||
name: "MeshCore PR Reviewer"
|
||||
description: "A specialized agent for reviewing pull requests in the meshcore-analyzer repository. It focuses on SOLID, DRY, testing, Go best practices, frontend testability, observability, and performance to prevent regressions and maintain high code quality."
|
||||
model: "gpt-5.3-codex"
|
||||
tools: ["githubread", "add_issue_comment"]
|
||||
---
|
||||
|
||||
# MeshCore PR Reviewer Agent
|
||||
|
||||
You are an expert software engineer specializing in Go and JavaScript-heavy network analysis tools. Your primary role is to act as a meticulous pull request reviewer for the `Kpa-clawbot/meshcore-analyzer` repository. You are deeply familiar with its architecture, as outlined in `AGENTS.md`, and you enforce its rules rigorously.
|
||||
|
||||
Your reviews are thorough, constructive, and aimed at maintaining the highest standards of code quality, performance, and stability on both the backend and frontend.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **Context is King**: Before any review, consult the `AGENTS.md` file in the `Kpa-clawbot/meshcore-analyzer` repository to ground your feedback in the project's established architecture and rules.
|
||||
2. **Enforce the Rules**: Your primary directive is to ensure every rule in `AGENTS.md` is followed. Call out any deviation.
|
||||
3. **Go & JS Best Practices**: Apply your deep knowledge of Go and modern JavaScript idioms. Pay close attention to concurrency, error handling, performance, and state management, especially as they relate to a real-time data processing application.
|
||||
4. **Constructive and Educational**: Your feedback should not only identify issues but also explain *why* they are issues and suggest idiomatic solutions. Your goal is to mentor and elevate the codebase and its contributors.
|
||||
5. **Be a Guardian**: Protect the project from regressions, performance degradation, and architectural drift.
|
||||
|
||||
## Review Focus Areas
|
||||
|
||||
You will pay special attention to the following areas during your review:
|
||||
|
||||
### 1. Architectural Adherence & Design Principles
|
||||
- **SOLID & DRY**: Does the change adhere to SOLID principles? Is there duplicated logic that could be refactored? Does it respect the existing separation of concerns?
|
||||
- **Project Architecture**: Does the PR respect the single Node.js server + static frontend architecture? Are changes in the right place?
|
||||
|
||||
### 2. Testing and Validation
|
||||
- **No commit without tests**: Is the backend logic change covered by unit tests? Is `test-packet-filter.js` or `test-aging.js` updated if necessary?
|
||||
- **Browser Validation**: Has the contributor confirmed the change works in a browser? Is there a screenshot for visual changes?
|
||||
- **Cache Busters**: If any `public/` assets (`.js`, `.css`) were modified, has the cache buster in `public/index.html` been bumped in the *same commit*? This is critical.
|
||||
|
||||
### 3. Go-Specific Concerns
|
||||
- **Concurrency**: Are goroutines used safely? Are there potential race conditions? Is synchronization used correctly?
|
||||
- **Error Handling**: Is error handling explicit and clear? Are errors wrapped with context where appropriate?
|
||||
- **Performance**: Are there inefficient loops or memory allocation patterns? Scrutinize any new data processing logic.
|
||||
- **Go Idioms**: Does the code follow standard Go idioms and formatting (`gofmt`)?
|
||||
|
||||
### 4. Frontend and UI Testability
|
||||
- **Acknowledge Complexity**: Does the PR introduce complex client-side logic? Recognize that browser-based functionality is difficult to unit test.
|
||||
- **Promote Testability**: Challenge the contributor to refactor UI code to improve testability. Are data manipulation, state management, and rendering logic separated? Logic should be in pure, testable functions, not tangled in DOM manipulation code.
|
||||
- **UI Logic Purity**: Scrutinize client-side JavaScript. Are there large, monolithic functions? Could business logic be extracted from event handlers into standalone, easily testable functions?
|
||||
- **State Management**: How is client-side state managed? Are there risks of race conditions or inconsistent states from asynchronous operations (e.g., API calls)?
|
||||
|
||||
### 5. Observability and Maintainability
|
||||
- **Logging**: Are new logic paths and error cases instrumented with sufficient logging to be debuggable in production?
|
||||
- **Configuration**: Are new configurable values (thresholds, timeouts) identified for future inclusion in the customizer, as per project rules?
|
||||
- **Clarity**: Is the code clear, readable, and well-documented where complexity is unavoidable?
|
||||
|
||||
### 6. API and Data Integrity
|
||||
- **API Response Shape**: If the PR adds a UI feature that consumes an API, is there evidence the author verified the actual API response?
|
||||
- **Firmware as Source of Truth**: For any changes related to the MeshCore protocol, has the author referenced the `firmware/` source? Challenge any "magic numbers" or assumptions about packet structure.
|
||||
|
||||
## Review Process
|
||||
|
||||
1. **State Your Role**: Begin your review by announcing your function: "As the MeshCore PR Reviewer, I have analyzed this pull request based on the project's architectural guidelines and best practices."
|
||||
2. **Provide a Summary**: Give a high-level summary of your findings (e.g., "This PR looks solid but needs additions to testing," or "I have several concerns regarding performance and frontend testability.").
|
||||
3. **Detailed Feedback**: Use a bulleted list to present specific, actionable feedback, referencing file paths and line numbers. For each point, cite the relevant principle or project rule (e.g., "Missing Test Coverage (Rule #1)", "UI Logic Purity (Focus Area #4)").
|
||||
4. **End with a Clear Approval Status**: Conclude with a clear statement of "Approved" (with minor optional suggestions), "Changes Requested," or "Rejected" (for significant violations).
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
name: "MeshCore PR Reviewer"
|
||||
description: "A specialized agent for reviewing pull requests in the meshcore-analyzer repository. It focuses on SOLID, DRY, testing, Go best practices, frontend testability, observability, and performance to prevent regressions and maintain high code quality."
|
||||
model: "gpt-5.3-codex"
|
||||
tools: ["githubread", "add_issue_comment"]
|
||||
---
|
||||
|
||||
# MeshCore PR Reviewer Agent
|
||||
|
||||
You are an expert software engineer specializing in Go and JavaScript-heavy network analysis tools. Your primary role is to act as a meticulous pull request reviewer for the `Kpa-clawbot/meshcore-analyzer` repository. You are deeply familiar with its architecture, as outlined in `AGENTS.md`, and you enforce its rules rigorously.
|
||||
|
||||
Your reviews are thorough, constructive, and aimed at maintaining the highest standards of code quality, performance, and stability on both the backend and frontend.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **Context is King**: Before any review, consult the `AGENTS.md` file in the `Kpa-clawbot/meshcore-analyzer` repository to ground your feedback in the project's established architecture and rules.
|
||||
2. **Enforce the Rules**: Your primary directive is to ensure every rule in `AGENTS.md` is followed. Call out any deviation.
|
||||
3. **Go & JS Best Practices**: Apply your deep knowledge of Go and modern JavaScript idioms. Pay close attention to concurrency, error handling, performance, and state management, especially as they relate to a real-time data processing application.
|
||||
4. **Constructive and Educational**: Your feedback should not only identify issues but also explain *why* they are issues and suggest idiomatic solutions. Your goal is to mentor and elevate the codebase and its contributors.
|
||||
5. **Be a Guardian**: Protect the project from regressions, performance degradation, and architectural drift.
|
||||
|
||||
## Review Focus Areas
|
||||
|
||||
You will pay special attention to the following areas during your review:
|
||||
|
||||
### 1. Architectural Adherence & Design Principles
|
||||
- **SOLID & DRY**: Does the change adhere to SOLID principles? Is there duplicated logic that could be refactored? Does it respect the existing separation of concerns?
|
||||
- **Project Architecture**: Does the PR respect the single Node.js server + static frontend architecture? Are changes in the right place?
|
||||
|
||||
### 2. Testing and Validation
|
||||
- **No commit without tests**: Is the backend logic change covered by unit tests? Is `test-packet-filter.js` or `test-aging.js` updated if necessary?
|
||||
- **Browser Validation**: Has the contributor confirmed the change works in a browser? Is there a screenshot for visual changes?
|
||||
- **Cache Busters**: If any `public/` assets (`.js`, `.css`) were modified, has the cache buster in `public/index.html` been bumped in the *same commit*? This is critical.
|
||||
|
||||
### 3. Go-Specific Concerns
|
||||
- **Concurrency**: Are goroutines used safely? Are there potential race conditions? Is synchronization used correctly?
|
||||
- **Error Handling**: Is error handling explicit and clear? Are errors wrapped with context where appropriate?
|
||||
- **Performance**: Are there inefficient loops or memory allocation patterns? Scrutinize any new data processing logic.
|
||||
- **Go Idioms**: Does the code follow standard Go idioms and formatting (`gofmt`)?
|
||||
|
||||
### 4. Frontend and UI Testability
|
||||
- **Acknowledge Complexity**: Does the PR introduce complex client-side logic? Recognize that browser-based functionality is difficult to unit test.
|
||||
- **Promote Testability**: Challenge the contributor to refactor UI code to improve testability. Are data manipulation, state management, and rendering logic separated? Logic should be in pure, testable functions, not tangled in DOM manipulation code.
|
||||
- **UI Logic Purity**: Scrutinize client-side JavaScript. Are there large, monolithic functions? Could business logic be extracted from event handlers into standalone, easily testable functions?
|
||||
- **State Management**: How is client-side state managed? Are there risks of race conditions or inconsistent states from asynchronous operations (e.g., API calls)?
|
||||
|
||||
### 5. Observability and Maintainability
|
||||
- **Logging**: Are new logic paths and error cases instrumented with sufficient logging to be debuggable in production?
|
||||
- **Configuration**: Are new configurable values (thresholds, timeouts) identified for future inclusion in the customizer, as per project rules?
|
||||
- **Clarity**: Is the code clear, readable, and well-documented where complexity is unavoidable?
|
||||
|
||||
### 6. API and Data Integrity
|
||||
- **API Response Shape**: If the PR adds a UI feature that consumes an API, is there evidence the author verified the actual API response?
|
||||
- **Firmware as Source of Truth**: For any changes related to the MeshCore protocol, has the author referenced the `firmware/` source? Challenge any "magic numbers" or assumptions about packet structure.
|
||||
|
||||
## Review Process
|
||||
|
||||
1. **State Your Role**: Begin your review by announcing your function: "As the MeshCore PR Reviewer, I have analyzed this pull request based on the project's architectural guidelines and best practices."
|
||||
2. **Provide a Summary**: Give a high-level summary of your findings (e.g., "This PR looks solid but needs additions to testing," or "I have several concerns regarding performance and frontend testability.").
|
||||
3. **Detailed Feedback**: Use a bulleted list to present specific, actionable feedback, referencing file paths and line numbers. For each point, cite the relevant principle or project rule (e.g., "Missing Test Coverage (Rule #1)", "UI Logic Purity (Focus Area #4)").
|
||||
4. **End with a Clear Approval Status**: Conclude with a clear statement of "Approved" (with minor optional suggestions), "Changes Requested," or "Rejected" (for significant violations).
|
||||
@@ -303,7 +303,7 @@ jobs:
|
||||
- name: Start staging on port 82
|
||||
run: |
|
||||
# Force remove stale containers
|
||||
docker rm -f meshcore-staging-go 2>/dev/null || true
|
||||
docker rm -f corescope-staging-go 2>/dev/null || true
|
||||
# Clean up stale ports
|
||||
fuser -k 82/tcp 2>/dev/null || true
|
||||
docker compose --profile staging-go up -d staging-go
|
||||
@@ -311,14 +311,14 @@ jobs:
|
||||
- name: Healthcheck staging container
|
||||
run: |
|
||||
for i in $(seq 1 120); do
|
||||
HEALTH=$(docker inspect meshcore-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
|
||||
HEALTH=$(docker inspect corescope-staging-go --format '{{.State.Health.Status}}' 2>/dev/null || echo "starting")
|
||||
if [ "$HEALTH" = "healthy" ]; then
|
||||
echo "Staging healthy after ${i}s"
|
||||
break
|
||||
fi
|
||||
if [ "$i" -eq 120 ]; then
|
||||
echo "Staging failed health check after 120s"
|
||||
docker logs meshcore-staging-go --tail 50
|
||||
docker logs corescope-staging-go --tail 50
|
||||
exit 1
|
||||
fi
|
||||
sleep 1
|
||||
@@ -378,6 +378,6 @@ jobs:
|
||||
echo "To promote to production:" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
|
||||
echo "ssh deploy@\$VM_HOST" >> $GITHUB_STEP_SUMMARY
|
||||
echo "cd /opt/meshcore-deploy" >> $GITHUB_STEP_SUMMARY
|
||||
echo "cd /opt/corescope-deploy" >> $GITHUB_STEP_SUMMARY
|
||||
echo "./manage.sh promote" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# AGENTS.md — MeshCore Analyzer
|
||||
# AGENTS.md — CoreScope
|
||||
|
||||
Guide for AI agents working on this codebase. Read this before writing any code.
|
||||
|
||||
|
||||
+6
-6
@@ -11,14 +11,14 @@ WORKDIR /build/server
|
||||
COPY cmd/server/go.mod cmd/server/go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/server/ ./
|
||||
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /meshcore-server .
|
||||
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /corescope-server .
|
||||
|
||||
# Build ingestor
|
||||
WORKDIR /build/ingestor
|
||||
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/ingestor/ ./
|
||||
RUN go build -o /meshcore-ingestor .
|
||||
RUN go build -o /corescope-ingestor .
|
||||
|
||||
# Runtime image
|
||||
FROM alpine:3.20
|
||||
@@ -28,15 +28,15 @@ RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
|
||||
WORKDIR /app
|
||||
|
||||
# Go binaries
|
||||
COPY --from=builder /meshcore-server /meshcore-ingestor /app/
|
||||
COPY --from=builder /corescope-server /corescope-ingestor /app/
|
||||
|
||||
# Frontend assets + config
|
||||
COPY public/ ./public/
|
||||
COPY config.example.json channel-rainbow.json ./
|
||||
|
||||
# Bake git commit SHA (CI writes .git-commit before build; fallback for non-ldflags usage)
|
||||
COPY .git-commi[t] ./
|
||||
RUN if [ ! -f .git-commit ]; then echo "unknown" > .git-commit; fi
|
||||
# Bake git commit SHA — manage.sh and CI write .git-commit before build
|
||||
# Default to "unknown" if not provided
|
||||
RUN echo "unknown" > .git-commit
|
||||
|
||||
# Supervisor + Mosquitto + Caddy config
|
||||
COPY docker/supervisord-go.conf /etc/supervisor/conf.d/supervisord.conf
|
||||
|
||||
+3
-3
@@ -11,14 +11,14 @@ WORKDIR /build/server
|
||||
COPY cmd/server/go.mod cmd/server/go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/server/ ./
|
||||
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /meshcore-server .
|
||||
RUN go build -ldflags "-X main.Version=${APP_VERSION} -X main.Commit=${GIT_COMMIT} -X main.BuildTime=${BUILD_TIME}" -o /corescope-server .
|
||||
|
||||
# Build ingestor
|
||||
WORKDIR /build/ingestor
|
||||
COPY cmd/ingestor/go.mod cmd/ingestor/go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/ingestor/ ./
|
||||
RUN go build -o /meshcore-ingestor .
|
||||
RUN go build -o /corescope-ingestor .
|
||||
|
||||
# Runtime image
|
||||
FROM alpine:3.20
|
||||
@@ -28,7 +28,7 @@ RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy wget
|
||||
WORKDIR /app
|
||||
|
||||
# Go binaries
|
||||
COPY --from=builder /meshcore-server /meshcore-ingestor /app/
|
||||
COPY --from=builder /corescope-server /corescope-ingestor /app/
|
||||
|
||||
# Frontend assets + config
|
||||
COPY public/ ./public/
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# MeshCore Analyzer
|
||||
# CoreScope
|
||||
|
||||
[](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/meshcore-analyzer/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
|
||||
[](https://github.com/Kpa-clawbot/corescope/actions/workflows/deploy.yml)
|
||||
|
||||
> High-performance mesh network analyzer powered by Go. Sub-millisecond packet queries, ~300 MB memory for 56K+ packets, real-time WebSocket broadcast, full channel decryption.
|
||||
|
||||
@@ -79,8 +79,8 @@ Full experience on your phone — proper touch controls, iOS safe area support,
|
||||
No Go installation needed — everything builds inside the container.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
|
||||
cd meshcore-analyzer
|
||||
git clone https://github.com/Kpa-clawbot/corescope.git
|
||||
cd corescope
|
||||
./manage.sh setup
|
||||
```
|
||||
|
||||
@@ -171,7 +171,7 @@ Or POST raw hex packets to `POST /api/packets` for manual injection.
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
meshcore-analyzer/
|
||||
corescope/
|
||||
├── cmd/
|
||||
│ ├── server/ # Go HTTP server + WebSocket + REST API
|
||||
│ │ ├── main.go # Entry point
|
||||
|
||||
+4
-4
@@ -73,8 +73,8 @@ Advert counts now reflect unique transmissions, not total observations. A packet
|
||||
|
||||
The Go backend is two binaries managed by supervisord inside Docker:
|
||||
|
||||
- **`meshcore-ingestor`** — connects to MQTT brokers, decodes packets, writes to SQLite, maintains the in-memory store
|
||||
- **`meshcore-server`** — HTTP API, WebSocket broadcast, static file serving, analytics computation
|
||||
- **`corescope-ingestor`** — connects to MQTT brokers, decodes packets, writes to SQLite, maintains the in-memory store
|
||||
- **`corescope-server`** — HTTP API, WebSocket broadcast, static file serving, analytics computation
|
||||
|
||||
Both share the same SQLite database (WAL mode). The frontend is unchanged — same vanilla JS, same `public/` directory, served by the Go HTTP server through Caddy.
|
||||
|
||||
@@ -120,7 +120,7 @@ curl -s http://localhost/api/health | grep engine
|
||||
The Node.js Dockerfile is preserved as `Dockerfile.node`:
|
||||
|
||||
```bash
|
||||
docker build -f Dockerfile.node -t meshcore-analyzer:latest .
|
||||
docker build -f Dockerfile.node -t corescope:latest .
|
||||
docker compose up -d --force-recreate prod
|
||||
```
|
||||
|
||||
@@ -152,7 +152,7 @@ This release wouldn't exist without the community:
|
||||
- **LitBomb** — issue reports from production deployments
|
||||
- **mibzzer15** — issue reports and edge case discovery
|
||||
|
||||
And to everyone running MeshCore Analyzer in the wild — your packet data, bug reports, and feature requests are what drive this project forward. The Go rewrite happened because the community outgrew what Node.js could handle. 56K packets, dozens of observers, sub-second queries. This is your tool. We just rewrote the engine.
|
||||
And to everyone running CoreScope in the wild — your packet data, bug reports, and feature requests are what drive this project forward. The Go rewrite happened because the community outgrew what Node.js could handle. 56K packets, dozens of observers, sub-second queries. This is your tool. We just rewrote the engine.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# MeshCore MQTT Ingestor (Go)
|
||||
|
||||
Standalone MQTT ingestion service for MeshCore Analyzer. Connects to MQTT brokers, decodes raw MeshCore packets, and writes to the same SQLite database used by the Node.js web server.
|
||||
Standalone MQTT ingestion service for CoreScope. Connects to MQTT brokers, decodes raw MeshCore packets, and writes to the same SQLite database used by the Node.js web server.
|
||||
|
||||
This is the first step of a larger Go rewrite — separating MQTT ingestion from the web server.
|
||||
|
||||
@@ -23,19 +23,19 @@ Requires Go 1.22+.
|
||||
|
||||
```bash
|
||||
cd cmd/ingestor
|
||||
go build -o meshcore-ingestor .
|
||||
go build -o corescope-ingestor .
|
||||
```
|
||||
|
||||
Cross-compile for Linux (e.g., for the production VM):
|
||||
|
||||
```bash
|
||||
GOOS=linux GOARCH=amd64 go build -o meshcore-ingestor .
|
||||
GOOS=linux GOARCH=amd64 go build -o corescope-ingestor .
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
./meshcore-ingestor -config /path/to/config.json
|
||||
./corescope-ingestor -config /path/to/config.json
|
||||
```
|
||||
|
||||
The config file uses the same format as the Node.js `config.json`. The ingestor reads the `mqttSources` array (or legacy `mqtt` object) and `dbPath` fields.
|
||||
|
||||
+156
-11
@@ -35,7 +35,8 @@ type Store struct {
|
||||
stmtUpsertNode *sql.Stmt
|
||||
stmtIncrementAdvertCount *sql.Stmt
|
||||
stmtUpsertObserver *sql.Stmt
|
||||
stmtGetObserverRowid *sql.Stmt
|
||||
stmtGetObserverRowid *sql.Stmt
|
||||
stmtUpdateNodeTelemetry *sql.Stmt
|
||||
}
|
||||
|
||||
// OpenStore opens or creates a SQLite DB at the given path, applying the
|
||||
@@ -81,7 +82,9 @@ func applySchema(db *sql.DB) error {
|
||||
lon REAL,
|
||||
last_seen TEXT,
|
||||
first_seen TEXT,
|
||||
advert_count INTEGER DEFAULT 0
|
||||
advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER,
|
||||
temperature_c REAL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS observers (
|
||||
@@ -97,7 +100,7 @@ func applySchema(db *sql.DB) error {
|
||||
radio TEXT,
|
||||
battery_mv INTEGER,
|
||||
uptime_secs INTEGER,
|
||||
noise_floor INTEGER
|
||||
noise_floor REAL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
|
||||
@@ -111,7 +114,9 @@ func applySchema(db *sql.DB) error {
|
||||
lon REAL,
|
||||
last_seen TEXT,
|
||||
first_seen TEXT,
|
||||
advert_count INTEGER DEFAULT 0
|
||||
advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER,
|
||||
temperature_c REAL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_inactive_nodes_last_seen ON inactive_nodes(last_seen);
|
||||
@@ -167,6 +172,25 @@ func applySchema(db *sql.DB) error {
|
||||
}
|
||||
}
|
||||
|
||||
// Create/rebuild packets_v view (v3 schema: observer_idx → observers.rowid)
|
||||
// The Go server reads this view; without it fresh installs get "no such table: packets_v".
|
||||
db.Exec(`DROP VIEW IF EXISTS packets_v`)
|
||||
_, vErr := db.Exec(`
|
||||
CREATE VIEW packets_v AS
|
||||
SELECT o.id, t.raw_hex,
|
||||
datetime(o.timestamp, 'unixepoch') AS timestamp,
|
||||
obs.id AS observer_id, obs.name AS observer_name,
|
||||
o.direction, o.snr, o.rssi, o.score, t.hash, t.route_type,
|
||||
t.payload_type, t.payload_version, o.path_json, t.decoded_json,
|
||||
t.created_at
|
||||
FROM observations o
|
||||
JOIN transmissions t ON t.id = o.transmission_id
|
||||
LEFT JOIN observers obs ON obs.rowid = o.observer_idx
|
||||
`)
|
||||
if vErr != nil {
|
||||
return fmt.Errorf("packets_v view: %w", vErr)
|
||||
}
|
||||
|
||||
// One-time migration: recalculate advert_count to count unique transmissions only
|
||||
db.Exec(`CREATE TABLE IF NOT EXISTS _migrations (name TEXT PRIMARY KEY)`)
|
||||
var migDone int
|
||||
@@ -184,6 +208,77 @@ func applySchema(db *sql.DB) error {
|
||||
log.Println("[migration] advert_count recalculated")
|
||||
}
|
||||
|
||||
// One-time migration: change noise_floor from INTEGER to REAL affinity.
|
||||
// SQLite doesn't support ALTER COLUMN, but existing float values are stored
|
||||
// as REAL regardless of column affinity. New table definition already uses REAL.
|
||||
// This migration casts any integer-stored noise_floor values to real.
|
||||
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'noise_floor_real_v1'")
|
||||
if row.Scan(&migDone) != nil {
|
||||
log.Println("[migration] Ensuring noise_floor values are stored as REAL...")
|
||||
db.Exec(`UPDATE observers SET noise_floor = CAST(noise_floor AS REAL) WHERE noise_floor IS NOT NULL AND typeof(noise_floor) = 'integer'`)
|
||||
db.Exec(`INSERT INTO _migrations (name) VALUES ('noise_floor_real_v1')`)
|
||||
log.Println("[migration] noise_floor migration complete")
|
||||
}
|
||||
|
||||
// One-time migration: add telemetry columns to nodes and inactive_nodes tables.
|
||||
row = db.QueryRow("SELECT 1 FROM _migrations WHERE name = 'node_telemetry_v1'")
|
||||
if row.Scan(&migDone) != nil {
|
||||
log.Println("[migration] Adding telemetry columns to nodes/inactive_nodes...")
|
||||
|
||||
// checkAndAddColumn checks whether `column` already exists in `table`
|
||||
// using PRAGMA table_info, and adds it if missing. All call sites pass
|
||||
// hardcoded table/column/type literals so there is no SQL injection risk.
|
||||
checkAndAddColumn := func(table, column, colType string) error {
|
||||
rows, err := db.Query(fmt.Sprintf("PRAGMA table_info(%s)", table))
|
||||
if err != nil {
|
||||
return fmt.Errorf("querying table info for %s: %w", table, err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
exists := false
|
||||
for rows.Next() {
|
||||
var cid int
|
||||
var name, ctype string
|
||||
var notnull, pk int
|
||||
var dfltValue sql.NullString
|
||||
if err := rows.Scan(&cid, &name, &ctype, ¬null, &dfltValue, &pk); err != nil {
|
||||
return fmt.Errorf("scanning table info for %s: %w", table, err)
|
||||
}
|
||||
if name == column {
|
||||
exists = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return fmt.Errorf("iterating table info for %s: %w", table, err)
|
||||
}
|
||||
if exists {
|
||||
return nil
|
||||
}
|
||||
if _, err := db.Exec(fmt.Sprintf("ALTER TABLE %s ADD COLUMN %s %s", table, column, colType)); err != nil {
|
||||
return fmt.Errorf("adding column %s to %s: %w", column, table, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
if err := checkAndAddColumn("nodes", "battery_mv", "INTEGER"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkAndAddColumn("nodes", "temperature_c", "REAL"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkAndAddColumn("inactive_nodes", "battery_mv", "INTEGER"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkAndAddColumn("inactive_nodes", "temperature_c", "REAL"); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := db.Exec(`INSERT INTO _migrations (name) VALUES ('node_telemetry_v1')`); err != nil {
|
||||
return fmt.Errorf("recording node_telemetry_v1 migration: %w", err)
|
||||
}
|
||||
log.Println("[migration] node telemetry columns added")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -238,13 +333,16 @@ func (s *Store) prepareStatements() error {
|
||||
}
|
||||
|
||||
s.stmtUpsertObserver, err = s.db.Prepare(`
|
||||
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
|
||||
VALUES (?, ?, ?, ?, ?, 1)
|
||||
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
|
||||
VALUES (?, ?, ?, ?, ?, 1, ?, ?, ?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
name = COALESCE(?, name),
|
||||
iata = COALESCE(?, iata),
|
||||
last_seen = ?,
|
||||
packet_count = packet_count + 1
|
||||
packet_count = packet_count + 1,
|
||||
battery_mv = COALESCE(?, battery_mv),
|
||||
uptime_secs = COALESCE(?, uptime_secs),
|
||||
noise_floor = COALESCE(?, noise_floor)
|
||||
`)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -255,6 +353,16 @@ func (s *Store) prepareStatements() error {
|
||||
return err
|
||||
}
|
||||
|
||||
s.stmtUpdateNodeTelemetry, err = s.db.Prepare(`
|
||||
UPDATE nodes SET
|
||||
battery_mv = COALESCE(?, battery_mv),
|
||||
temperature_c = COALESCE(?, temperature_c)
|
||||
WHERE public_key = ?
|
||||
`)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -359,12 +467,49 @@ func (s *Store) IncrementAdvertCount(pubKey string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// UpsertObserver inserts or updates an observer.
|
||||
func (s *Store) UpsertObserver(id, name, iata string) error {
|
||||
// UpdateNodeTelemetry updates battery and temperature for a node.
|
||||
func (s *Store) UpdateNodeTelemetry(pubKey string, batteryMv *int, temperatureC *float64) error {
|
||||
var bv, tc interface{}
|
||||
if batteryMv != nil {
|
||||
bv = *batteryMv
|
||||
}
|
||||
if temperatureC != nil {
|
||||
tc = *temperatureC
|
||||
}
|
||||
_, err := s.stmtUpdateNodeTelemetry.Exec(bv, tc, pubKey)
|
||||
if err != nil {
|
||||
s.Stats.WriteErrors.Add(1)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// ObserverMeta holds optional observer hardware metadata.
|
||||
type ObserverMeta struct {
|
||||
BatteryMv *int // millivolts, always integer
|
||||
UptimeSecs *int64 // seconds, always integer
|
||||
NoiseFloor *float64 // dBm, may have decimals
|
||||
}
|
||||
|
||||
// UpsertObserver inserts or updates an observer with optional hardware metadata.
|
||||
func (s *Store) UpsertObserver(id, name, iata string, meta *ObserverMeta) error {
|
||||
now := time.Now().UTC().Format(time.RFC3339)
|
||||
|
||||
var batteryMv, uptimeSecs, noiseFloor interface{}
|
||||
if meta != nil {
|
||||
if meta.BatteryMv != nil {
|
||||
batteryMv = *meta.BatteryMv
|
||||
}
|
||||
if meta.UptimeSecs != nil {
|
||||
uptimeSecs = *meta.UptimeSecs
|
||||
}
|
||||
if meta.NoiseFloor != nil {
|
||||
noiseFloor = *meta.NoiseFloor
|
||||
}
|
||||
}
|
||||
|
||||
_, err := s.stmtUpsertObserver.Exec(
|
||||
id, name, iata, now, now,
|
||||
name, iata, now,
|
||||
id, name, iata, now, now, batteryMv, uptimeSecs, noiseFloor,
|
||||
name, iata, now, batteryMv, uptimeSecs, noiseFloor,
|
||||
)
|
||||
if err != nil {
|
||||
s.Stats.WriteErrors.Add(1)
|
||||
|
||||
+297
-7
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -62,6 +63,16 @@ func TestOpenStore(t *testing.T) {
|
||||
t.Errorf("missing table %s, got %v", e, tables)
|
||||
}
|
||||
}
|
||||
|
||||
// Verify packets_v view exists
|
||||
var viewCount int
|
||||
err = s.db.QueryRow("SELECT COUNT(*) FROM sqlite_master WHERE type='view' AND name='packets_v'").Scan(&viewCount)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if viewCount != 1 {
|
||||
t.Error("packets_v view not created")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInsertTransmission(t *testing.T) {
|
||||
@@ -114,6 +125,54 @@ func TestInsertTransmission(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestPacketsViewQueryable(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
// Insert observer so the LEFT JOIN resolves
|
||||
if err := s.UpsertObserver("obs1", "TestObserver", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
snr := 3.5
|
||||
rssi := -95.0
|
||||
data := &PacketData{
|
||||
RawHex: "AABB",
|
||||
Timestamp: "2026-01-01T00:00:00Z",
|
||||
ObserverID: "obs1",
|
||||
Hash: "viewtesthash",
|
||||
RouteType: 1,
|
||||
PayloadType: 4,
|
||||
PathJSON: "[]",
|
||||
DecodedJSON: `{"type":"ADVERT"}`,
|
||||
SNR: &snr,
|
||||
RSSI: &rssi,
|
||||
}
|
||||
if _, err := s.InsertTransmission(data); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Query through packets_v — the view the Go server relies on
|
||||
var obsID, obsName sql.NullString
|
||||
var hash string
|
||||
err = s.db.QueryRow("SELECT observer_id, observer_name, hash FROM packets_v LIMIT 1").Scan(&obsID, &obsName, &hash)
|
||||
if err != nil {
|
||||
t.Fatalf("packets_v query failed: %v", err)
|
||||
}
|
||||
if hash != "viewtesthash" {
|
||||
t.Errorf("hash=%s, want viewtesthash", hash)
|
||||
}
|
||||
if !obsID.Valid || obsID.String != "obs1" {
|
||||
t.Errorf("observer_id=%v, want obs1", obsID)
|
||||
}
|
||||
if !obsName.Valid || obsName.String != "TestObserver" {
|
||||
t.Errorf("observer_name=%v, want TestObserver", obsName)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertNode(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
@@ -160,7 +219,7 @@ func TestUpsertObserver(t *testing.T) {
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -174,6 +233,165 @@ func TestUpsertObserver(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertObserverWithMeta(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
battery := 3500
|
||||
uptime := int64(86400)
|
||||
noise := -115.5
|
||||
meta := &ObserverMeta{
|
||||
BatteryMv: &battery,
|
||||
UptimeSecs: &uptime,
|
||||
NoiseFloor: &noise,
|
||||
}
|
||||
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", meta); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Verify correct types in DB
|
||||
var batteryMv int
|
||||
var uptimeSecs int64
|
||||
var noiseFloor float64
|
||||
err = s.db.QueryRow("SELECT battery_mv, uptime_secs, noise_floor FROM observers WHERE id = 'obs1'").
|
||||
Scan(&batteryMv, &uptimeSecs, &noiseFloor)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if batteryMv != 3500 {
|
||||
t.Errorf("battery_mv=%d, want 3500", batteryMv)
|
||||
}
|
||||
if uptimeSecs != 86400 {
|
||||
t.Errorf("uptime_secs=%d, want 86400", uptimeSecs)
|
||||
}
|
||||
if noiseFloor != -115.5 {
|
||||
t.Errorf("noise_floor=%f, want -115.5", noiseFloor)
|
||||
}
|
||||
|
||||
// Verify typeof returns correct SQLite types
|
||||
var typBattery, typUptime, typNoise string
|
||||
s.db.QueryRow("SELECT typeof(battery_mv), typeof(uptime_secs), typeof(noise_floor) FROM observers WHERE id = 'obs1'").
|
||||
Scan(&typBattery, &typUptime, &typNoise)
|
||||
if typBattery != "integer" {
|
||||
t.Errorf("typeof(battery_mv)=%s, want integer", typBattery)
|
||||
}
|
||||
if typUptime != "integer" {
|
||||
t.Errorf("typeof(uptime_secs)=%s, want integer", typUptime)
|
||||
}
|
||||
if typNoise != "real" {
|
||||
t.Errorf("typeof(noise_floor)=%s, want real", typNoise)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertObserverMetaPreservesExisting(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
// First upsert with metadata
|
||||
battery := 3500
|
||||
noise := -115.5
|
||||
meta := &ObserverMeta{
|
||||
BatteryMv: &battery,
|
||||
NoiseFloor: &noise,
|
||||
}
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", meta); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Second upsert without metadata — should preserve existing values
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var batteryMv int
|
||||
var noiseFloor float64
|
||||
s.db.QueryRow("SELECT battery_mv, noise_floor FROM observers WHERE id = 'obs1'").
|
||||
Scan(&batteryMv, &noiseFloor)
|
||||
if batteryMv != 3500 {
|
||||
t.Errorf("battery_mv=%d after nil-meta upsert, want 3500 (preserved)", batteryMv)
|
||||
}
|
||||
if noiseFloor != -115.5 {
|
||||
t.Errorf("noise_floor=%f after nil-meta upsert, want -115.5 (preserved)", noiseFloor)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractObserverMeta(t *testing.T) {
|
||||
// Float values from JSON (typical MQTT payload)
|
||||
msg := map[string]interface{}{
|
||||
"battery_mv": 3500.0,
|
||||
"uptime_secs": 86400.0,
|
||||
"noise_floor": -115.5,
|
||||
}
|
||||
meta := extractObserverMeta(msg)
|
||||
if meta == nil {
|
||||
t.Fatal("expected non-nil meta")
|
||||
}
|
||||
if meta.BatteryMv == nil || *meta.BatteryMv != 3500 {
|
||||
t.Errorf("BatteryMv=%v, want 3500", meta.BatteryMv)
|
||||
}
|
||||
if meta.UptimeSecs == nil || *meta.UptimeSecs != 86400 {
|
||||
t.Errorf("UptimeSecs=%v, want 86400", meta.UptimeSecs)
|
||||
}
|
||||
if meta.NoiseFloor == nil || *meta.NoiseFloor != -115.5 {
|
||||
t.Errorf("NoiseFloor=%v, want -115.5", meta.NoiseFloor)
|
||||
}
|
||||
|
||||
// Battery with fractional part should round
|
||||
msg2 := map[string]interface{}{
|
||||
"battery_mv": 3500.7,
|
||||
}
|
||||
meta2 := extractObserverMeta(msg2)
|
||||
if meta2 == nil || meta2.BatteryMv == nil || *meta2.BatteryMv != 3501 {
|
||||
t.Errorf("battery_mv rounding: got %v, want 3501", meta2)
|
||||
}
|
||||
|
||||
// Empty message → nil
|
||||
meta3 := extractObserverMeta(map[string]interface{}{})
|
||||
if meta3 != nil {
|
||||
t.Errorf("expected nil for empty message, got %v", meta3)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSchemaNoiseFloorIsReal(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
// Check column type affinity via PRAGMA
|
||||
rows, err := s.db.Query("PRAGMA table_info(observers)")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
for rows.Next() {
|
||||
var cid int
|
||||
var colName, colType string
|
||||
var notNull, pk int
|
||||
var dflt interface{}
|
||||
if rows.Scan(&cid, &colName, &colType, ¬Null, &dflt, &pk) == nil {
|
||||
if colName == "noise_floor" && colType != "REAL" {
|
||||
t.Errorf("noise_floor column type=%s, want REAL", colType)
|
||||
}
|
||||
if colName == "battery_mv" && colType != "INTEGER" {
|
||||
t.Errorf("battery_mv column type=%s, want INTEGER", colType)
|
||||
}
|
||||
if colName == "uptime_secs" && colType != "INTEGER" {
|
||||
t.Errorf("uptime_secs column type=%s, want INTEGER", colType)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInsertTransmissionWithObserver(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
@@ -182,7 +400,7 @@ func TestInsertTransmissionWithObserver(t *testing.T) {
|
||||
defer s.Close()
|
||||
|
||||
// Insert observer first
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -639,7 +857,7 @@ func TestConcurrentWrites(t *testing.T) {
|
||||
defer s.Close()
|
||||
|
||||
// Pre-create an observer for observer_idx resolution
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -681,7 +899,7 @@ func TestConcurrentWrites(t *testing.T) {
|
||||
return
|
||||
}
|
||||
obsID := fmt.Sprintf("obs_%d_%d__________", gIdx, i)
|
||||
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC", nil); err != nil {
|
||||
errCh <- fmt.Errorf("goroutine %d observer upsert %d: %w", gIdx, i, err)
|
||||
return
|
||||
}
|
||||
@@ -782,7 +1000,7 @@ func TestDBStats(t *testing.T) {
|
||||
}
|
||||
|
||||
// Observer upsert
|
||||
if err := s.UpsertObserver("obs1", "Obs1", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver("obs1", "Obs1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s.Stats.ObserverUpserts.Load() != 1 {
|
||||
@@ -801,7 +1019,7 @@ func TestLoadTestThroughput(t *testing.T) {
|
||||
defer s.Close()
|
||||
|
||||
// Pre-create observer
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver("obs1", "Observer1", "SJC", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -867,7 +1085,7 @@ func TestLoadTestThroughput(t *testing.T) {
|
||||
}
|
||||
|
||||
obsID := fmt.Sprintf("obs_%04d_%04d_____", gIdx, i)
|
||||
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC"); err != nil {
|
||||
if err := s.UpsertObserver(obsID[:16], "Obs", "SJC", nil); err != nil {
|
||||
totalErrors.Add(1)
|
||||
if strings.Contains(err.Error(), "locked") || strings.Contains(err.Error(), "BUSY") {
|
||||
busyErrors.Add(1)
|
||||
@@ -933,3 +1151,75 @@ func TestLoadTestThroughput(t *testing.T) {
|
||||
t.Errorf("transmissions=%d, want %d", txCount, totalMessages)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateNodeTelemetry(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
lat := 37.0
|
||||
lon := -122.0
|
||||
if err := s.UpsertNode("telem1", "TelemetryNode", "sensor", &lat, &lon, "2026-03-25T00:00:00Z"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
battery := 3700
|
||||
temp := 28.5
|
||||
if err := s.UpdateNodeTelemetry("telem1", &battery, &temp); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var bv int
|
||||
var tc float64
|
||||
err = s.db.QueryRow("SELECT battery_mv, temperature_c FROM nodes WHERE public_key = 'telem1'").Scan(&bv, &tc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if bv != 3700 {
|
||||
t.Errorf("battery_mv=%d, want 3700", bv)
|
||||
}
|
||||
if tc != 28.5 {
|
||||
t.Errorf("temperature_c=%f, want 28.5", tc)
|
||||
}
|
||||
|
||||
newTemp := -5.0
|
||||
if err := s.UpdateNodeTelemetry("telem1", nil, &newTemp); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err = s.db.QueryRow("SELECT battery_mv, temperature_c FROM nodes WHERE public_key = 'telem1'").Scan(&bv, &tc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if bv != 3700 {
|
||||
t.Errorf("battery_mv after nil update=%d, want 3700 (preserved)", bv)
|
||||
}
|
||||
if tc != -5.0 {
|
||||
t.Errorf("temperature_c after update=%f, want -5.0", tc)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTelemetryMigrationAddsColumns(t *testing.T) {
|
||||
s, err := OpenStore(tempDBPath(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer s.Close()
|
||||
|
||||
_, err = s.db.Exec("SELECT battery_mv, temperature_c FROM nodes LIMIT 1")
|
||||
if err != nil {
|
||||
t.Errorf("nodes table should have battery_mv and temperature_c columns: %v", err)
|
||||
}
|
||||
|
||||
_, err = s.db.Exec("SELECT battery_mv, temperature_c FROM inactive_nodes LIMIT 1")
|
||||
if err != nil {
|
||||
t.Errorf("inactive_nodes table should have battery_mv and temperature_c columns: %v", err)
|
||||
}
|
||||
|
||||
var count int
|
||||
s.db.QueryRow("SELECT COUNT(*) FROM _migrations WHERE name = 'node_telemetry_v1'").Scan(&count)
|
||||
if count != 1 {
|
||||
t.Errorf("migration node_telemetry_v1 should be recorded, count=%d", count)
|
||||
}
|
||||
}
|
||||
|
||||
+31
-2
@@ -111,6 +111,8 @@ type Payload struct {
|
||||
Lat *float64 `json:"lat,omitempty"`
|
||||
Lon *float64 `json:"lon,omitempty"`
|
||||
Name string `json:"name,omitempty"`
|
||||
BatteryMv *int `json:"battery_mv,omitempty"`
|
||||
TemperatureC *float64 `json:"temperature_c,omitempty"`
|
||||
ChannelHash int `json:"channelHash,omitempty"`
|
||||
ChannelHashHex string `json:"channelHashHex,omitempty"`
|
||||
DecryptionStatus string `json:"decryptionStatus,omitempty"`
|
||||
@@ -251,10 +253,37 @@ func decodeAdvert(buf []byte) Payload {
|
||||
off += 8
|
||||
}
|
||||
if p.Flags.HasName {
|
||||
name := string(appdata[off:])
|
||||
name = strings.TrimRight(name, "\x00")
|
||||
// Find null terminator to separate name from trailing telemetry bytes
|
||||
nameEnd := len(appdata)
|
||||
for i := off; i < len(appdata); i++ {
|
||||
if appdata[i] == 0x00 {
|
||||
nameEnd = i
|
||||
break
|
||||
}
|
||||
}
|
||||
name := string(appdata[off:nameEnd])
|
||||
name = sanitizeName(name)
|
||||
p.Name = name
|
||||
off = nameEnd
|
||||
// Skip null terminator(s)
|
||||
for off < len(appdata) && appdata[off] == 0x00 {
|
||||
off++
|
||||
}
|
||||
}
|
||||
|
||||
// Telemetry bytes after name: battery_mv(2 LE) + temperature_c(2 LE, signed, /100)
|
||||
// Only sensor nodes (advType=4) carry telemetry bytes.
|
||||
if p.Flags.Sensor && off+4 <= len(appdata) {
|
||||
batteryMv := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
|
||||
tempRaw := int16(binary.LittleEndian.Uint16(appdata[off+2 : off+4]))
|
||||
tempC := float64(tempRaw) / 100.0
|
||||
if batteryMv > 0 && batteryMv <= 10000 {
|
||||
p.BatteryMv = &batteryMv
|
||||
}
|
||||
// Raw int16 / 100 → °C; accept -50°C to 100°C (raw: -5000 to 10000)
|
||||
if tempRaw >= -5000 && tempRaw <= 10000 {
|
||||
p.TemperatureC = &tempC
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1355,3 +1355,154 @@ func TestDecodeGrpTxtGarbageMarkedFailed(t *testing.T) {
|
||||
t.Errorf("type=%s, want GRP_TXT", p.Type)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertWithTelemetry(t *testing.T) {
|
||||
pubkey := strings.Repeat("AA", 32)
|
||||
timestamp := "78563412"
|
||||
signature := strings.Repeat("BB", 64)
|
||||
flags := "94" // sensor(4) | hasLocation(0x10) | hasName(0x80)
|
||||
lat := "40933402"
|
||||
lon := "E0E6B8F8"
|
||||
name := hex.EncodeToString([]byte("Sensor1"))
|
||||
nullTerm := "00"
|
||||
batteryLE := make([]byte, 2)
|
||||
binary.LittleEndian.PutUint16(batteryLE, 3700)
|
||||
tempLE := make([]byte, 2)
|
||||
binary.LittleEndian.PutUint16(tempLE, uint16(int16(2850)))
|
||||
|
||||
hexStr := "1200" + pubkey + timestamp + signature + flags + lat + lon +
|
||||
name + nullTerm +
|
||||
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
|
||||
|
||||
pkt, err := DecodePacket(hexStr, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if pkt.Payload.Name != "Sensor1" {
|
||||
t.Errorf("name=%s, want Sensor1", pkt.Payload.Name)
|
||||
}
|
||||
if pkt.Payload.BatteryMv == nil {
|
||||
t.Fatal("battery_mv should not be nil")
|
||||
}
|
||||
if *pkt.Payload.BatteryMv != 3700 {
|
||||
t.Errorf("battery_mv=%d, want 3700", *pkt.Payload.BatteryMv)
|
||||
}
|
||||
if pkt.Payload.TemperatureC == nil {
|
||||
t.Fatal("temperature_c should not be nil")
|
||||
}
|
||||
if math.Abs(*pkt.Payload.TemperatureC-28.50) > 0.01 {
|
||||
t.Errorf("temperature_c=%f, want 28.50", *pkt.Payload.TemperatureC)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertWithTelemetryNegativeTemp(t *testing.T) {
|
||||
pubkey := strings.Repeat("CC", 32)
|
||||
timestamp := "00000000"
|
||||
signature := strings.Repeat("DD", 64)
|
||||
flags := "84" // sensor(4) | hasName(0x80), no location
|
||||
name := hex.EncodeToString([]byte("Cold"))
|
||||
nullTerm := "00"
|
||||
batteryLE := make([]byte, 2)
|
||||
binary.LittleEndian.PutUint16(batteryLE, 4200)
|
||||
tempLE := make([]byte, 2)
|
||||
var negTemp int16 = -550
|
||||
binary.LittleEndian.PutUint16(tempLE, uint16(negTemp))
|
||||
|
||||
hexStr := "1200" + pubkey + timestamp + signature + flags +
|
||||
name + nullTerm +
|
||||
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
|
||||
|
||||
pkt, err := DecodePacket(hexStr, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if pkt.Payload.Name != "Cold" {
|
||||
t.Errorf("name=%s, want Cold", pkt.Payload.Name)
|
||||
}
|
||||
if pkt.Payload.BatteryMv == nil || *pkt.Payload.BatteryMv != 4200 {
|
||||
t.Errorf("battery_mv=%v, want 4200", pkt.Payload.BatteryMv)
|
||||
}
|
||||
if pkt.Payload.TemperatureC == nil {
|
||||
t.Fatal("temperature_c should not be nil")
|
||||
}
|
||||
if math.Abs(*pkt.Payload.TemperatureC-(-5.50)) > 0.01 {
|
||||
t.Errorf("temperature_c=%f, want -5.50", *pkt.Payload.TemperatureC)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertWithoutTelemetry(t *testing.T) {
|
||||
pubkey := strings.Repeat("EE", 32)
|
||||
timestamp := "00000000"
|
||||
signature := strings.Repeat("FF", 64)
|
||||
flags := "82" // repeater(2) | hasName(0x80)
|
||||
name := hex.EncodeToString([]byte("Node1"))
|
||||
|
||||
hexStr := "1200" + pubkey + timestamp + signature + flags + name
|
||||
pkt, err := DecodePacket(hexStr, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if pkt.Payload.Name != "Node1" {
|
||||
t.Errorf("name=%s, want Node1", pkt.Payload.Name)
|
||||
}
|
||||
if pkt.Payload.BatteryMv != nil {
|
||||
t.Errorf("battery_mv should be nil for advert without telemetry, got %d", *pkt.Payload.BatteryMv)
|
||||
}
|
||||
if pkt.Payload.TemperatureC != nil {
|
||||
t.Errorf("temperature_c should be nil for advert without telemetry, got %f", *pkt.Payload.TemperatureC)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertNonSensorIgnoresTelemetryBytes(t *testing.T) {
|
||||
// A repeater node with 4 trailing bytes after the name should NOT decode telemetry.
|
||||
pubkey := strings.Repeat("AB", 32)
|
||||
timestamp := "00000000"
|
||||
signature := strings.Repeat("CD", 64)
|
||||
flags := "82" // repeater(2) | hasName(0x80)
|
||||
name := hex.EncodeToString([]byte("Rptr"))
|
||||
nullTerm := "00"
|
||||
extraBytes := "B40ED403" // battery-like and temp-like bytes
|
||||
|
||||
hexStr := "1200" + pubkey + timestamp + signature + flags + name + nullTerm + extraBytes
|
||||
pkt, err := DecodePacket(hexStr, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if pkt.Payload.BatteryMv != nil {
|
||||
t.Errorf("battery_mv should be nil for non-sensor node, got %d", *pkt.Payload.BatteryMv)
|
||||
}
|
||||
if pkt.Payload.TemperatureC != nil {
|
||||
t.Errorf("temperature_c should be nil for non-sensor node, got %f", *pkt.Payload.TemperatureC)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertTelemetryZeroTemp(t *testing.T) {
|
||||
// 0°C is a valid temperature and must be emitted.
|
||||
pubkey := strings.Repeat("12", 32)
|
||||
timestamp := "00000000"
|
||||
signature := strings.Repeat("34", 64)
|
||||
flags := "84" // sensor(4) | hasName(0x80)
|
||||
name := hex.EncodeToString([]byte("FreezeSensor"))
|
||||
nullTerm := "00"
|
||||
batteryLE := make([]byte, 2)
|
||||
binary.LittleEndian.PutUint16(batteryLE, 3600)
|
||||
tempLE := make([]byte, 2) // tempRaw=0 → 0°C
|
||||
|
||||
hexStr := "1200" + pubkey + timestamp + signature + flags +
|
||||
name + nullTerm +
|
||||
hex.EncodeToString(batteryLE) + hex.EncodeToString(tempLE)
|
||||
|
||||
pkt, err := DecodePacket(hexStr, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if pkt.Payload.TemperatureC == nil {
|
||||
t.Fatal("temperature_c should not be nil for 0°C")
|
||||
}
|
||||
if *pkt.Payload.TemperatureC != 0.0 {
|
||||
t.Errorf("temperature_c=%f, want 0.0", *pkt.Payload.TemperatureC)
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
module github.com/meshcore-analyzer/ingestor
|
||||
module github.com/corescope/ingestor
|
||||
|
||||
go 1.22
|
||||
|
||||
|
||||
+60
-3
@@ -8,6 +8,9 @@ import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"log"
|
||||
"math"
|
||||
"net/http"
|
||||
_ "net/http/pprof"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
@@ -19,6 +22,20 @@ import (
|
||||
)
|
||||
|
||||
func main() {
|
||||
// pprof profiling — off by default, enable with ENABLE_PPROF=true
|
||||
if os.Getenv("ENABLE_PPROF") == "true" {
|
||||
pprofPort := os.Getenv("PPROF_PORT")
|
||||
if pprofPort == "" {
|
||||
pprofPort = "6061"
|
||||
}
|
||||
go func() {
|
||||
log.Printf("[pprof] ingestor profiling at http://localhost:%s/debug/pprof/", pprofPort)
|
||||
if err := http.ListenAndServe(":"+pprofPort, nil); err != nil {
|
||||
log.Printf("[pprof] failed to start: %v (non-fatal)", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
configPath := flag.String("config", "config.json", "path to config file")
|
||||
flag.Parse()
|
||||
|
||||
@@ -193,7 +210,8 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
|
||||
observerID := parts[2]
|
||||
name, _ := msg["origin"].(string)
|
||||
iata := parts[1]
|
||||
if err := store.UpsertObserver(observerID, name, iata); err != nil {
|
||||
meta := extractObserverMeta(msg)
|
||||
if err := store.UpsertObserver(observerID, name, iata, meta); err != nil {
|
||||
log.Printf("MQTT [%s] observer status error: %v", tag, err)
|
||||
}
|
||||
log.Printf("MQTT [%s] status: %s (%s)", tag, firstNonEmpty(name, observerID), iata)
|
||||
@@ -252,6 +270,12 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
|
||||
log.Printf("MQTT [%s] advert count error: %v", tag, err)
|
||||
}
|
||||
}
|
||||
// Update telemetry if present in advert
|
||||
if decoded.Payload.BatteryMv != nil || decoded.Payload.TemperatureC != nil {
|
||||
if err := store.UpdateNodeTelemetry(decoded.Payload.PubKey, decoded.Payload.BatteryMv, decoded.Payload.TemperatureC); err != nil {
|
||||
log.Printf("MQTT [%s] node telemetry update error: %v", tag, err)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
log.Printf("MQTT [%s] skipping corrupted ADVERT: %s", tag, reason)
|
||||
}
|
||||
@@ -260,7 +284,7 @@ func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message,
|
||||
// Upsert observer
|
||||
if observerID != "" {
|
||||
origin, _ := msg["origin"].(string)
|
||||
if err := store.UpsertObserver(observerID, origin, region); err != nil {
|
||||
if err := store.UpsertObserver(observerID, origin, region, nil); err != nil {
|
||||
log.Printf("MQTT [%s] observer upsert error: %v", tag, err)
|
||||
}
|
||||
}
|
||||
@@ -446,6 +470,39 @@ func toFloat64(v interface{}) (float64, bool) {
|
||||
}
|
||||
}
|
||||
|
||||
// extractObserverMeta extracts hardware metadata from an MQTT status message.
|
||||
// Casts battery_mv and uptime_secs to integers (they're always whole numbers).
|
||||
func extractObserverMeta(msg map[string]interface{}) *ObserverMeta {
|
||||
meta := &ObserverMeta{}
|
||||
hasData := false
|
||||
|
||||
if v, ok := msg["battery_mv"]; ok {
|
||||
if f, ok := toFloat64(v); ok {
|
||||
iv := int(math.Round(f))
|
||||
meta.BatteryMv = &iv
|
||||
hasData = true
|
||||
}
|
||||
}
|
||||
if v, ok := msg["uptime_secs"]; ok {
|
||||
if f, ok := toFloat64(v); ok {
|
||||
iv := int64(math.Round(f))
|
||||
meta.UptimeSecs = &iv
|
||||
hasData = true
|
||||
}
|
||||
}
|
||||
if v, ok := msg["noise_floor"]; ok {
|
||||
if f, ok := toFloat64(v); ok {
|
||||
meta.NoiseFloor = &f
|
||||
hasData = true
|
||||
}
|
||||
}
|
||||
|
||||
if !hasData {
|
||||
return nil
|
||||
}
|
||||
return meta
|
||||
}
|
||||
|
||||
func firstNonEmpty(vals ...string) string {
|
||||
for _, v := range vals {
|
||||
if v != "" {
|
||||
@@ -495,7 +552,7 @@ var version = "dev"
|
||||
|
||||
func init() {
|
||||
if len(os.Args) > 1 && os.Args[1] == "--version" {
|
||||
fmt.Println("meshcore-ingestor", version)
|
||||
fmt.Println("corescope-ingestor", version)
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,12 +26,13 @@ func setupTestDBv2(t *testing.T) *DB {
|
||||
schema := `
|
||||
CREATE TABLE nodes (
|
||||
public_key TEXT PRIMARY KEY, name TEXT, role TEXT,
|
||||
lat REAL, lon REAL, last_seen TEXT, first_seen TEXT, advert_count INTEGER DEFAULT 0
|
||||
lat REAL, lon REAL, last_seen TEXT, first_seen TEXT, advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER, temperature_c REAL
|
||||
);
|
||||
CREATE TABLE observers (
|
||||
id TEXT PRIMARY KEY, name TEXT, iata TEXT, last_seen TEXT, first_seen TEXT,
|
||||
packet_count INTEGER DEFAULT 0, model TEXT, firmware TEXT,
|
||||
client_version TEXT, radio TEXT, battery_mv INTEGER, uptime_secs INTEGER, noise_floor INTEGER
|
||||
client_version TEXT, radio TEXT, battery_mv INTEGER, uptime_secs INTEGER, noise_floor REAL
|
||||
);
|
||||
CREATE TABLE transmissions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT, raw_hex TEXT NOT NULL,
|
||||
|
||||
+59
-20
@@ -128,23 +128,25 @@ type Node struct {
|
||||
LastSeen *string `json:"last_seen"`
|
||||
FirstSeen *string `json:"first_seen"`
|
||||
AdvertCount int `json:"advert_count"`
|
||||
BatteryMv *int `json:"battery_mv"`
|
||||
TemperatureC *float64 `json:"temperature_c"`
|
||||
}
|
||||
|
||||
// Observer represents a row from the observers table.
|
||||
type Observer struct {
|
||||
ID string `json:"id"`
|
||||
Name *string `json:"name"`
|
||||
IATA *string `json:"iata"`
|
||||
LastSeen *string `json:"last_seen"`
|
||||
FirstSeen *string `json:"first_seen"`
|
||||
PacketCount int `json:"packet_count"`
|
||||
Model *string `json:"model"`
|
||||
Firmware *string `json:"firmware"`
|
||||
ClientVersion *string `json:"client_version"`
|
||||
Radio *string `json:"radio"`
|
||||
BatteryMv *int `json:"battery_mv"`
|
||||
UptimeSecs *int `json:"uptime_secs"`
|
||||
NoiseFloor *int `json:"noise_floor"`
|
||||
ID string `json:"id"`
|
||||
Name *string `json:"name"`
|
||||
IATA *string `json:"iata"`
|
||||
LastSeen *string `json:"last_seen"`
|
||||
FirstSeen *string `json:"first_seen"`
|
||||
PacketCount int `json:"packet_count"`
|
||||
Model *string `json:"model"`
|
||||
Firmware *string `json:"firmware"`
|
||||
ClientVersion *string `json:"client_version"`
|
||||
Radio *string `json:"radio"`
|
||||
BatteryMv *int `json:"battery_mv"`
|
||||
UptimeSecs *int64 `json:"uptime_secs"`
|
||||
NoiseFloor *float64 `json:"noise_floor"`
|
||||
}
|
||||
|
||||
// Transmission represents a row from the transmissions table.
|
||||
@@ -739,7 +741,7 @@ func (db *DB) GetNodes(limit, offset int, role, search, before, lastHeard, sortB
|
||||
var total int
|
||||
db.conn.QueryRow(fmt.Sprintf("SELECT COUNT(*) FROM nodes %s", w), args...).Scan(&total)
|
||||
|
||||
querySQL := fmt.Sprintf("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count FROM nodes %s ORDER BY %s LIMIT ? OFFSET ?", w, order)
|
||||
querySQL := fmt.Sprintf("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c FROM nodes %s ORDER BY %s LIMIT ? OFFSET ?", w, order)
|
||||
qArgs := append(args, limit, offset)
|
||||
|
||||
rows, err := db.conn.Query(querySQL, qArgs...)
|
||||
@@ -765,7 +767,7 @@ func (db *DB) SearchNodes(query string, limit int) ([]map[string]interface{}, er
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
rows, err := db.conn.Query(`SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count
|
||||
rows, err := db.conn.Query(`SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c
|
||||
FROM nodes WHERE name LIKE ? OR public_key LIKE ? ORDER BY last_seen DESC LIMIT ?`,
|
||||
"%"+query+"%", query+"%", limit)
|
||||
if err != nil {
|
||||
@@ -785,7 +787,7 @@ func (db *DB) SearchNodes(query string, limit int) ([]map[string]interface{}, er
|
||||
|
||||
// GetNodeByPubkey returns a single node.
|
||||
func (db *DB) GetNodeByPubkey(pubkey string) (map[string]interface{}, error) {
|
||||
rows, err := db.conn.Query("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count FROM nodes WHERE public_key = ?", pubkey)
|
||||
rows, err := db.conn.Query("SELECT public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c FROM nodes WHERE public_key = ?", pubkey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -958,9 +960,21 @@ func (db *DB) GetObservers() ([]Observer, error) {
|
||||
var observers []Observer
|
||||
for rows.Next() {
|
||||
var o Observer
|
||||
if err := rows.Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &o.BatteryMv, &o.UptimeSecs, &o.NoiseFloor); err != nil {
|
||||
var batteryMv, uptimeSecs sql.NullInt64
|
||||
var noiseFloor sql.NullFloat64
|
||||
if err := rows.Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor); err != nil {
|
||||
continue
|
||||
}
|
||||
if batteryMv.Valid {
|
||||
v := int(batteryMv.Int64)
|
||||
o.BatteryMv = &v
|
||||
}
|
||||
if uptimeSecs.Valid {
|
||||
o.UptimeSecs = &uptimeSecs.Int64
|
||||
}
|
||||
if noiseFloor.Valid {
|
||||
o.NoiseFloor = &noiseFloor.Float64
|
||||
}
|
||||
observers = append(observers, o)
|
||||
}
|
||||
return observers, nil
|
||||
@@ -969,11 +983,23 @@ func (db *DB) GetObservers() ([]Observer, error) {
|
||||
// GetObserverByID returns a single observer.
|
||||
func (db *DB) GetObserverByID(id string) (*Observer, error) {
|
||||
var o Observer
|
||||
var batteryMv, uptimeSecs sql.NullInt64
|
||||
var noiseFloor sql.NullFloat64
|
||||
err := db.conn.QueryRow("SELECT id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor FROM observers WHERE id = ?", id).
|
||||
Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &o.BatteryMv, &o.UptimeSecs, &o.NoiseFloor)
|
||||
Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount, &o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if batteryMv.Valid {
|
||||
v := int(batteryMv.Int64)
|
||||
o.BatteryMv = &v
|
||||
}
|
||||
if uptimeSecs.Valid {
|
||||
o.UptimeSecs = &uptimeSecs.Int64
|
||||
}
|
||||
if noiseFloor.Valid {
|
||||
o.NoiseFloor = &noiseFloor.Float64
|
||||
}
|
||||
return &o, nil
|
||||
}
|
||||
|
||||
@@ -1634,11 +1660,13 @@ func scanNodeRow(rows *sql.Rows) map[string]interface{} {
|
||||
var name, role, lastSeen, firstSeen sql.NullString
|
||||
var lat, lon sql.NullFloat64
|
||||
var advertCount int
|
||||
var batteryMv sql.NullInt64
|
||||
var temperatureC sql.NullFloat64
|
||||
|
||||
if err := rows.Scan(&pk, &name, &role, &lat, &lon, &lastSeen, &firstSeen, &advertCount); err != nil {
|
||||
if err := rows.Scan(&pk, &name, &role, &lat, &lon, &lastSeen, &firstSeen, &advertCount, &batteryMv, &temperatureC); err != nil {
|
||||
return nil
|
||||
}
|
||||
return map[string]interface{}{
|
||||
m := map[string]interface{}{
|
||||
"public_key": pk,
|
||||
"name": nullStr(name),
|
||||
"role": nullStr(role),
|
||||
@@ -1651,6 +1679,17 @@ func scanNodeRow(rows *sql.Rows) map[string]interface{} {
|
||||
"hash_size": nil,
|
||||
"hash_size_inconsistent": false,
|
||||
}
|
||||
if batteryMv.Valid {
|
||||
m["battery_mv"] = int(batteryMv.Int64)
|
||||
} else {
|
||||
m["battery_mv"] = nil
|
||||
}
|
||||
if temperatureC.Valid {
|
||||
m["temperature_c"] = temperatureC.Float64
|
||||
} else {
|
||||
m["temperature_c"] = nil
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func nullStr(ns sql.NullString) interface{} {
|
||||
|
||||
+129
-2
@@ -28,7 +28,9 @@ func setupTestDB(t *testing.T) *DB {
|
||||
lon REAL,
|
||||
last_seen TEXT,
|
||||
first_seen TEXT,
|
||||
advert_count INTEGER DEFAULT 0
|
||||
advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER,
|
||||
temperature_c REAL
|
||||
);
|
||||
|
||||
CREATE TABLE observers (
|
||||
@@ -44,7 +46,7 @@ func setupTestDB(t *testing.T) *DB {
|
||||
radio TEXT,
|
||||
battery_mv INTEGER,
|
||||
uptime_secs INTEGER,
|
||||
noise_floor INTEGER
|
||||
noise_floor REAL
|
||||
);
|
||||
|
||||
CREATE TABLE transmissions (
|
||||
@@ -369,6 +371,88 @@ func TestGetObserverByIDNotFound(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestObserverTypeConsistency(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
// Insert observer with typed metadata matching ingestor writes
|
||||
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
|
||||
VALUES ('obs_typed', 'TypedObs', 'SJC', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 10, 3500, 86400, -115.5)`)
|
||||
|
||||
obs, err := db.GetObserverByID("obs_typed")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// battery_mv should be *int
|
||||
if obs.BatteryMv == nil {
|
||||
t.Fatal("BatteryMv should not be nil")
|
||||
}
|
||||
if *obs.BatteryMv != 3500 {
|
||||
t.Errorf("BatteryMv=%d, want 3500", *obs.BatteryMv)
|
||||
}
|
||||
|
||||
// uptime_secs should be *int64
|
||||
if obs.UptimeSecs == nil {
|
||||
t.Fatal("UptimeSecs should not be nil")
|
||||
}
|
||||
if *obs.UptimeSecs != 86400 {
|
||||
t.Errorf("UptimeSecs=%d, want 86400", *obs.UptimeSecs)
|
||||
}
|
||||
|
||||
// noise_floor should be *float64
|
||||
if obs.NoiseFloor == nil {
|
||||
t.Fatal("NoiseFloor should not be nil")
|
||||
}
|
||||
if *obs.NoiseFloor != -115.5 {
|
||||
t.Errorf("NoiseFloor=%f, want -115.5", *obs.NoiseFloor)
|
||||
}
|
||||
|
||||
// Verify NULL handling: observer without metadata
|
||||
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
|
||||
VALUES ('obs_null', 'NullObs', 'SFO', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 5)`)
|
||||
|
||||
obsNull, err := db.GetObserverByID("obs_null")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if obsNull.BatteryMv != nil {
|
||||
t.Errorf("BatteryMv should be nil for observer without metadata, got %d", *obsNull.BatteryMv)
|
||||
}
|
||||
if obsNull.UptimeSecs != nil {
|
||||
t.Errorf("UptimeSecs should be nil for observer without metadata, got %d", *obsNull.UptimeSecs)
|
||||
}
|
||||
if obsNull.NoiseFloor != nil {
|
||||
t.Errorf("NoiseFloor should be nil for observer without metadata, got %f", *obsNull.NoiseFloor)
|
||||
}
|
||||
}
|
||||
|
||||
func TestObserverTypesInGetObservers(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec(`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, battery_mv, uptime_secs, noise_floor)
|
||||
VALUES ('obs1', 'Obs1', 'SJC', '2026-06-01T00:00:00Z', '2026-01-01T00:00:00Z', 10, 4200, 172800, -110.3)`)
|
||||
|
||||
observers, err := db.GetObservers()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(observers) != 1 {
|
||||
t.Fatalf("expected 1 observer, got %d", len(observers))
|
||||
}
|
||||
o := observers[0]
|
||||
if o.BatteryMv == nil || *o.BatteryMv != 4200 {
|
||||
t.Errorf("BatteryMv=%v, want 4200", o.BatteryMv)
|
||||
}
|
||||
if o.UptimeSecs == nil || *o.UptimeSecs != 172800 {
|
||||
t.Errorf("UptimeSecs=%v, want 172800", o.UptimeSecs)
|
||||
}
|
||||
if o.NoiseFloor == nil || *o.NoiseFloor != -110.3 {
|
||||
t.Errorf("NoiseFloor=%v, want -110.3", o.NoiseFloor)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetDistinctIATAs(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
@@ -1386,6 +1470,49 @@ func TestGetChannelsStaleMessage(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestNodeTelemetryFields(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen, advert_count, battery_mv, temperature_c)
|
||||
VALUES ('pk_telem1', 'SensorNode', 'sensor', 37.0, -122.0, '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 5, 3700, 28.5)`)
|
||||
|
||||
node, err := db.GetNodeByPubkey("pk_telem1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if node == nil {
|
||||
t.Fatal("expected node, got nil")
|
||||
}
|
||||
if node["battery_mv"] != 3700 {
|
||||
t.Errorf("battery_mv=%v, want 3700", node["battery_mv"])
|
||||
}
|
||||
if node["temperature_c"] != 28.5 {
|
||||
t.Errorf("temperature_c=%v, want 28.5", node["temperature_c"])
|
||||
}
|
||||
|
||||
nodes, _, _, err := db.GetNodes(50, 0, "sensor", "", "", "", "", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(nodes) != 1 {
|
||||
t.Fatalf("expected 1 sensor node, got %d", len(nodes))
|
||||
}
|
||||
if nodes[0]["battery_mv"] != 3700 {
|
||||
t.Errorf("GetNodes battery_mv=%v, want 3700", nodes[0]["battery_mv"])
|
||||
}
|
||||
|
||||
db.conn.Exec(`INSERT INTO nodes (public_key, name, role, last_seen, first_seen, advert_count)
|
||||
VALUES ('pk_notelem', 'PlainNode', 'repeater', '2026-01-01T00:00:00Z', '2026-01-01T00:00:00Z', 3)`)
|
||||
node2, _ := db.GetNodeByPubkey("pk_notelem")
|
||||
if node2["battery_mv"] != nil {
|
||||
t.Errorf("expected nil battery_mv for node without telemetry, got %v", node2["battery_mv"])
|
||||
}
|
||||
if node2["temperature_c"] != nil {
|
||||
t.Errorf("expected nil temperature_c for node without telemetry, got %v", node2["temperature_c"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestMain(m *testing.M) {
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
module github.com/meshcore-analyzer/server
|
||||
module github.com/corescope/server
|
||||
|
||||
go 1.22
|
||||
|
||||
|
||||
+18
-3
@@ -6,6 +6,7 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
"net/http"
|
||||
_ "net/http/pprof"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/signal"
|
||||
@@ -54,6 +55,20 @@ func resolveBuildTime() string {
|
||||
}
|
||||
|
||||
func main() {
|
||||
// pprof profiling — off by default, enable with ENABLE_PPROF=true
|
||||
if os.Getenv("ENABLE_PPROF") == "true" {
|
||||
pprofPort := os.Getenv("PPROF_PORT")
|
||||
if pprofPort == "" {
|
||||
pprofPort = "6060"
|
||||
}
|
||||
go func() {
|
||||
log.Printf("[pprof] profiling UI at http://localhost:%s/debug/pprof/", pprofPort)
|
||||
if err := http.ListenAndServe(":"+pprofPort, nil); err != nil {
|
||||
log.Printf("[pprof] failed to start: %v (non-fatal)", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
var (
|
||||
configDir string
|
||||
port int
|
||||
@@ -101,7 +116,7 @@ func main() {
|
||||
var tableName string
|
||||
err = database.conn.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name='transmissions'").Scan(&tableName)
|
||||
if err == sql.ErrNoRows {
|
||||
log.Fatalf("[db] table 'transmissions' not found — is this a MeshCore Analyzer database?")
|
||||
log.Fatalf("[db] table 'transmissions' not found — is this a CoreScope database?")
|
||||
}
|
||||
|
||||
stats, err := database.GetStats()
|
||||
@@ -140,7 +155,7 @@ func main() {
|
||||
log.Printf("[static] directory %s not found — API-only mode", absPublic)
|
||||
router.PathPrefix("/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/html")
|
||||
w.Write([]byte(`<!DOCTYPE html><html><body><h1>MeshCore Analyzer</h1><p>Frontend not found. API available at /api/</p></body></html>`))
|
||||
w.Write([]byte(`<!DOCTYPE html><html><body><h1>CoreScope</h1><p>Frontend not found. API available at /api/</p></body></html>`))
|
||||
})
|
||||
}
|
||||
|
||||
@@ -167,7 +182,7 @@ func main() {
|
||||
httpServer.Close()
|
||||
}()
|
||||
|
||||
log.Printf("[server] MeshCore Analyzer (Go) listening on http://localhost:%d", cfg.Port)
|
||||
log.Printf("[server] CoreScope (Go) listening on http://localhost:%d", cfg.Port)
|
||||
if err := httpServer.ListenAndServe(); err != http.ErrServerClosed {
|
||||
log.Fatalf("[server] %v", err)
|
||||
}
|
||||
|
||||
@@ -240,7 +240,7 @@ func (s *Server) handleConfigTheme(w http.ResponseWriter, r *http.Request) {
|
||||
theme := LoadTheme(".")
|
||||
|
||||
branding := mergeMap(map[string]interface{}{
|
||||
"siteName": "MeshCore Analyzer",
|
||||
"siteName": "CoreScope",
|
||||
"tagline": "Real-time MeshCore LoRa mesh network analyzer",
|
||||
}, s.cfg.Branding, theme.Branding)
|
||||
|
||||
|
||||
+12
@@ -996,6 +996,12 @@
|
||||
"elementShape": {
|
||||
"type": "number"
|
||||
}
|
||||
},
|
||||
"battery_mv": {
|
||||
"type": "nullable_number"
|
||||
},
|
||||
"temperature_c": {
|
||||
"type": "nullable_number"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -1097,6 +1103,12 @@
|
||||
},
|
||||
"last_heard": {
|
||||
"type": "string"
|
||||
},
|
||||
"battery_mv": {
|
||||
"type": "nullable_number"
|
||||
},
|
||||
"temperature_c": {
|
||||
"type": "nullable_number"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -10,7 +10,7 @@
|
||||
"key": "/path/to/key.pem"
|
||||
},
|
||||
"branding": {
|
||||
"siteName": "MeshCore Analyzer",
|
||||
"siteName": "CoreScope",
|
||||
"tagline": "Real-time MeshCore LoRa mesh network analyzer",
|
||||
"logoUrl": null,
|
||||
"faviconUrl": null
|
||||
@@ -32,7 +32,7 @@
|
||||
"observer": "#8b5cf6"
|
||||
},
|
||||
"home": {
|
||||
"heroTitle": "MeshCore Analyzer",
|
||||
"heroTitle": "CoreScope",
|
||||
"heroSubtitle": "Find your nodes to start monitoring them.",
|
||||
"steps": [
|
||||
{ "emoji": "📡", "title": "Connect", "description": "Link your node to the mesh" },
|
||||
|
||||
@@ -33,7 +33,9 @@ db.exec(`
|
||||
lon REAL,
|
||||
last_seen TEXT,
|
||||
first_seen TEXT,
|
||||
advert_count INTEGER DEFAULT 0
|
||||
advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER,
|
||||
temperature_c REAL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS observers (
|
||||
@@ -60,7 +62,9 @@ db.exec(`
|
||||
lon REAL,
|
||||
last_seen TEXT,
|
||||
first_seen TEXT,
|
||||
advert_count INTEGER DEFAULT 0
|
||||
advert_count INTEGER DEFAULT 0,
|
||||
battery_mv INTEGER,
|
||||
temperature_c REAL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
|
||||
@@ -324,6 +328,22 @@ for (const col of ['model', 'firmware', 'client_version', 'radio', 'battery_mv',
|
||||
}
|
||||
}
|
||||
|
||||
// --- One-time migration: add telemetry columns to nodes and inactive_nodes ---
|
||||
{
|
||||
const done = db.prepare(`SELECT 1 FROM _migrations WHERE name = 'node_telemetry_v1'`).get();
|
||||
if (!done) {
|
||||
console.log('[migration] Adding telemetry columns to nodes/inactive_nodes...');
|
||||
const nodeCols = db.pragma('table_info(nodes)').map(c => c.name);
|
||||
if (!nodeCols.includes('battery_mv')) db.exec(`ALTER TABLE nodes ADD COLUMN battery_mv INTEGER`);
|
||||
if (!nodeCols.includes('temperature_c')) db.exec(`ALTER TABLE nodes ADD COLUMN temperature_c REAL`);
|
||||
const inactiveCols = db.pragma('table_info(inactive_nodes)').map(c => c.name);
|
||||
if (!inactiveCols.includes('battery_mv')) db.exec(`ALTER TABLE inactive_nodes ADD COLUMN battery_mv INTEGER`);
|
||||
if (!inactiveCols.includes('temperature_c')) db.exec(`ALTER TABLE inactive_nodes ADD COLUMN temperature_c REAL`);
|
||||
db.prepare(`INSERT INTO _migrations (name) VALUES ('node_telemetry_v1')`).run();
|
||||
console.log('[migration] node telemetry columns added');
|
||||
}
|
||||
}
|
||||
|
||||
// --- Prepared statements ---
|
||||
const stmts = {
|
||||
upsertNode: db.prepare(`
|
||||
@@ -339,6 +359,12 @@ const stmts = {
|
||||
incrementAdvertCount: db.prepare(`
|
||||
UPDATE nodes SET advert_count = advert_count + 1 WHERE public_key = @public_key
|
||||
`),
|
||||
updateNodeTelemetry: db.prepare(`
|
||||
UPDATE nodes SET
|
||||
battery_mv = COALESCE(@battery_mv, battery_mv),
|
||||
temperature_c = COALESCE(@temperature_c, temperature_c)
|
||||
WHERE public_key = @public_key
|
||||
`),
|
||||
upsertObserver: db.prepare(`
|
||||
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor)
|
||||
VALUES (@id, @name, @iata, @last_seen, @first_seen, 1, @model, @firmware, @client_version, @radio, @battery_mv, @uptime_secs, @noise_floor)
|
||||
@@ -511,6 +537,14 @@ function incrementAdvertCount(publicKey) {
|
||||
stmts.incrementAdvertCount.run({ public_key: publicKey });
|
||||
}
|
||||
|
||||
function updateNodeTelemetry(data) {
|
||||
stmts.updateNodeTelemetry.run({
|
||||
public_key: data.public_key,
|
||||
battery_mv: data.battery_mv ?? null,
|
||||
temperature_c: data.temperature_c ?? null,
|
||||
});
|
||||
}
|
||||
|
||||
function upsertNode(data) {
|
||||
const now = new Date().toISOString();
|
||||
stmts.upsertNode.run({
|
||||
@@ -898,4 +932,4 @@ function moveStaleNodes(nodeDays) {
|
||||
return moved;
|
||||
}
|
||||
|
||||
module.exports = { db, schemaVersion, observerIdToRowid, resolveObserverIdx, insertTransmission, upsertNode, incrementAdvertCount, upsertObserver, updateObserverStatus, getPackets, getPacket, getTransmission, getNodes, getNode, getObservers, getStats, searchNodes, getNodeHealth, getNodeAnalytics, removePhantomNodes, moveStaleNodes };
|
||||
module.exports = { db, schemaVersion, observerIdToRowid, resolveObserverIdx, insertTransmission, upsertNode, incrementAdvertCount, updateNodeTelemetry, upsertObserver, updateObserverStatus, getPackets, getPacket, getTransmission, getNodes, getNode, getObservers, getStats, searchNodes, getNodeHealth, getNodeAnalytics, removePhantomNodes, moveStaleNodes };
|
||||
|
||||
+24
-2
@@ -135,10 +135,32 @@ function decodeAdvert(buf) {
|
||||
off += 8;
|
||||
}
|
||||
if (result.flags.hasName) {
|
||||
let name = appdata.subarray(off).toString('utf8');
|
||||
// Strip non-printable characters (< 0x20 except tab/newline) and DEL
|
||||
// Find null terminator to separate name from trailing telemetry bytes
|
||||
let nameEnd = appdata.length;
|
||||
for (let i = off; i < appdata.length; i++) {
|
||||
if (appdata[i] === 0x00) { nameEnd = i; break; }
|
||||
}
|
||||
let name = appdata.subarray(off, nameEnd).toString('utf8');
|
||||
name = name.replace(/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/g, '');
|
||||
result.name = name;
|
||||
off = nameEnd;
|
||||
// Skip null terminator(s)
|
||||
while (off < appdata.length && appdata[off] === 0x00) off++;
|
||||
}
|
||||
|
||||
// Telemetry bytes after name: battery_mv(2 LE) + temperature_c(2 LE, signed, /100)
|
||||
// Only sensor nodes (advType=4) carry telemetry bytes.
|
||||
if (result.flags.sensor && off + 4 <= appdata.length) {
|
||||
const batteryMv = appdata.readUInt16LE(off);
|
||||
const tempRaw = appdata.readInt16LE(off + 2);
|
||||
const tempC = tempRaw / 100.0;
|
||||
if (batteryMv > 0 && batteryMv <= 10000) {
|
||||
result.battery_mv = batteryMv;
|
||||
}
|
||||
// Raw int16 / 100 → °C; accept -50°C to 100°C (raw: -5000 to 10000)
|
||||
if (tempRaw >= -5000 && tempRaw <= 10000) {
|
||||
result.temperature_c = tempC;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+9
-6
@@ -3,8 +3,8 @@
|
||||
|
||||
services:
|
||||
prod:
|
||||
image: meshcore-analyzer:latest
|
||||
container_name: meshcore-prod
|
||||
image: corescope:latest
|
||||
container_name: corescope-prod
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${PROD_HTTP_PORT:-80}:${PROD_HTTP_PORT:-80}"
|
||||
@@ -24,8 +24,8 @@ services:
|
||||
retries: 3
|
||||
|
||||
staging:
|
||||
image: meshcore-analyzer:latest
|
||||
container_name: meshcore-staging
|
||||
image: corescope:latest
|
||||
container_name: corescope-staging
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${STAGING_HTTP_PORT:-81}:${STAGING_HTTP_PORT:-81}"
|
||||
@@ -52,18 +52,21 @@ services:
|
||||
args:
|
||||
APP_VERSION: ${APP_VERSION:-unknown}
|
||||
GIT_COMMIT: ${GIT_COMMIT:-unknown}
|
||||
image: meshcore-go:latest
|
||||
container_name: meshcore-staging-go
|
||||
image: corescope-go:latest
|
||||
container_name: corescope-staging-go
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${STAGING_GO_HTTP_PORT:-82}:80"
|
||||
- "${STAGING_GO_MQTT_PORT:-1885}:1883"
|
||||
- "6060:6060" # pprof server
|
||||
- "6061:6061" # pprof ingestor
|
||||
volumes:
|
||||
- ${STAGING_DATA_DIR:-~/meshcore-staging-data}/config.json:/app/config.json:ro
|
||||
- ${STAGING_DATA_DIR:-~/meshcore-staging-data}:/app/data
|
||||
- caddy-data-staging-go:/data/caddy
|
||||
environment:
|
||||
- NODE_ENV=staging
|
||||
- ENABLE_PPROF=true
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/stats"]
|
||||
interval: 30s
|
||||
|
||||
@@ -14,8 +14,8 @@ stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:meshcore-ingestor]
|
||||
command=/app/meshcore-ingestor -config /app/config.json
|
||||
[program:corescope-ingestor]
|
||||
command=/app/corescope-ingestor -config /app/config.json
|
||||
directory=/app
|
||||
autostart=true
|
||||
autorestart=true
|
||||
@@ -24,8 +24,8 @@ stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:meshcore-server]
|
||||
command=/app/meshcore-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
|
||||
[program:corescope-server]
|
||||
command=/app/corescope-server -config-dir /app -db /app/data/meshcore.db -public /app/public -port 3000
|
||||
directory=/app
|
||||
autostart=true
|
||||
autorestart=true
|
||||
|
||||
@@ -14,7 +14,7 @@ stdout_logfile_maxbytes=0
|
||||
stderr_logfile=/dev/stderr
|
||||
stderr_logfile_maxbytes=0
|
||||
|
||||
[program:meshcore-analyzer]
|
||||
[program:corescope]
|
||||
command=node /app/server.js
|
||||
directory=/app
|
||||
autostart=true
|
||||
|
||||
@@ -27,7 +27,7 @@ No restart needed. The server picks up changes to `theme.json` on every page loa
|
||||
**Bare metal / PM2 / systemd:**
|
||||
```bash
|
||||
# Same directory as server.js and config.json
|
||||
cp theme.json /path/to/meshcore-analyzer/
|
||||
cp theme.json /path/to/corescope/
|
||||
```
|
||||
|
||||
Check the server logs on startup — it tells you where it's looking:
|
||||
|
||||
+22
-22
@@ -1,6 +1,6 @@
|
||||
# Deploying MeshCore Analyzer
|
||||
# Deploying CoreScope
|
||||
|
||||
Get MeshCore Analyzer running with automatic HTTPS on your own server.
|
||||
Get CoreScope running with automatic HTTPS on your own server.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -19,7 +19,7 @@ Get MeshCore Analyzer running with automatic HTTPS on your own server.
|
||||
|
||||
## What You'll End Up With
|
||||
|
||||
- MeshCore Analyzer running at `https://your-domain.com`
|
||||
- CoreScope running at `https://your-domain.com`
|
||||
- Automatic HTTPS certificates (via Let's Encrypt + Caddy)
|
||||
- Built-in MQTT broker for receiving packets from observers
|
||||
- SQLite database for packet storage (auto-created)
|
||||
@@ -83,8 +83,8 @@ docker --version
|
||||
The easiest way — use the management script:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
|
||||
cd meshcore-analyzer
|
||||
git clone https://github.com/Kpa-clawbot/corescope.git
|
||||
cd corescope
|
||||
./manage.sh setup
|
||||
```
|
||||
|
||||
@@ -111,8 +111,8 @@ flowchart LR
|
||||
### 1. Download the code
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
|
||||
cd meshcore-analyzer
|
||||
git clone https://github.com/Kpa-clawbot/corescope.git
|
||||
cd corescope
|
||||
```
|
||||
|
||||
### 2. Create your config
|
||||
@@ -153,10 +153,10 @@ Save and close. Caddy handles certificates, renewals, and HTTP→HTTPS redirects
|
||||
### 4. Build and run
|
||||
|
||||
```bash
|
||||
docker build -t meshcore-analyzer .
|
||||
docker build -t corescope .
|
||||
|
||||
docker run -d \
|
||||
--name meshcore-analyzer \
|
||||
--name corescope \
|
||||
--restart unless-stopped \
|
||||
-p 80:80 \
|
||||
-p 443:443 \
|
||||
@@ -164,7 +164,7 @@ docker run -d \
|
||||
-v $(pwd)/caddy-config/Caddyfile:/etc/caddy/Caddyfile:ro \
|
||||
-v meshcore-data:/app/data \
|
||||
-v caddy-data:/data/caddy \
|
||||
meshcore-analyzer
|
||||
corescope
|
||||
```
|
||||
|
||||
What each flag does:
|
||||
@@ -184,12 +184,12 @@ Open `https://your-domain.com`. You should see the analyzer home page.
|
||||
|
||||
Check the logs:
|
||||
```bash
|
||||
docker logs meshcore-analyzer
|
||||
docker logs corescope
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
MeshCore Analyzer running on http://localhost:3000
|
||||
CoreScope running on http://localhost:3000
|
||||
MQTT [local] connected to mqtt://localhost:1883
|
||||
[pre-warm] 12 endpoints in XXXms
|
||||
```
|
||||
@@ -215,7 +215,7 @@ Add a remote broker to `mqttSources` in your `config.json`:
|
||||
}
|
||||
```
|
||||
|
||||
Restart: `docker restart meshcore-analyzer`
|
||||
Restart: `docker restart corescope`
|
||||
|
||||
### Option B: Run your own observer
|
||||
|
||||
@@ -271,12 +271,12 @@ If you already run a reverse proxy, skip Caddy entirely and proxy directly to th
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name meshcore-analyzer \
|
||||
--name corescope \
|
||||
--restart unless-stopped \
|
||||
-p 3000:3000 \
|
||||
-v $(pwd)/config.json:/app/config.json:ro \
|
||||
-v meshcore-data:/app/data \
|
||||
meshcore-analyzer
|
||||
corescope
|
||||
```
|
||||
|
||||
Then configure your existing proxy to forward traffic to `localhost:3000`.
|
||||
@@ -287,12 +287,12 @@ For local testing or a LAN-only setup, use the default Caddyfile that ships in t
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name meshcore-analyzer \
|
||||
--name corescope \
|
||||
--restart unless-stopped \
|
||||
-p 80:80 \
|
||||
-v $(pwd)/config.json:/app/config.json:ro \
|
||||
-v meshcore-data:/app/data \
|
||||
meshcore-analyzer
|
||||
corescope
|
||||
```
|
||||
|
||||
## MQTT Security
|
||||
@@ -315,7 +315,7 @@ password_file /etc/mosquitto/passwd
|
||||
```
|
||||
After starting the container, create users:
|
||||
```bash
|
||||
docker exec -it meshcore-analyzer mosquitto_passwd -c /etc/mosquitto/passwd myuser
|
||||
docker exec -it corescope mosquitto_passwd -c /etc/mosquitto/passwd myuser
|
||||
```
|
||||
|
||||
**Option 3: Use TLS** — For production, configure Mosquitto with TLS certificates. See the [Mosquitto docs](https://mosquitto.org/man/mosquitto-conf-5.html).
|
||||
@@ -331,7 +331,7 @@ Packet data is stored in `meshcore.db` inside the data volume.
|
||||
**Using manage.sh (easiest):**
|
||||
|
||||
```bash
|
||||
./manage.sh backup # Saves to ./backups/meshcore-TIMESTAMP.db
|
||||
./manage.sh backup # Saves to ./backups/corescope-TIMESTAMP/
|
||||
./manage.sh backup ~/my-backup.db # Custom path
|
||||
./manage.sh restore ./backups/some-file.db # Restore (backs up current DB first)
|
||||
```
|
||||
@@ -345,7 +345,7 @@ If you used `-v ./analyzer-data:/app/data` instead of a Docker volume, the datab
|
||||
```bash
|
||||
crontab -e
|
||||
# Add:
|
||||
0 3 * * * cd /path/to/meshcore-analyzer && ./manage.sh backup
|
||||
0 3 * * * cd /path/to/corescope && ./manage.sh backup
|
||||
```
|
||||
|
||||
## Updating
|
||||
@@ -398,11 +398,11 @@ Center the map on your area in `config.json`:
|
||||
|
||||
| Problem | Likely cause | Fix |
|
||||
|---------|-------------|-----|
|
||||
| Site shows "connection refused" | Container not running | `docker ps` to check, `docker logs meshcore-analyzer` for errors |
|
||||
| Site shows "connection refused" | Container not running | `docker ps` to check, `docker logs corescope` for errors |
|
||||
| HTTPS not working | Port 80 blocked | Open port 80 — Caddy needs it for ACME challenges |
|
||||
| "too many certificates" error | Let's Encrypt rate limit (5/domain/week) | Use a different subdomain, bring your own cert, or wait a week |
|
||||
| Certificate won't provision | DNS not pointed at server | `dig your-domain` must show your server IP before starting |
|
||||
| No packets appearing | No observer connected | `docker exec meshcore-analyzer mosquitto_sub -t 'meshcore/#' -C 1 -W 10` — if silent, no data is coming in |
|
||||
| No packets appearing | No observer connected | `docker exec corescope mosquitto_sub -t 'meshcore/#' -C 1 -W 10` — if silent, no data is coming in |
|
||||
| Container crashes on startup | Bad JSON in config | `python3 -c "import json; json.load(open('config.json'))"` to validate |
|
||||
| "address already in use" | Another web server on 80/443 | Stop it: `sudo systemctl stop nginx apache2` |
|
||||
| Slow on Raspberry Pi | First build is slow | Normal — subsequent builds use cache. Runtime performance is fine. |
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Hash Prefix Disambiguation in MeshCore Analyzer
|
||||
# Hash Prefix Disambiguation in CoreScope
|
||||
|
||||
## Section 1: Executive Summary
|
||||
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
# MeshCore Analyzer — API Contract Specification
|
||||
# CoreScope — API Contract Specification
|
||||
|
||||
> **Authoritative contract.** Both the Node.js and Go backends MUST conform to this spec.
|
||||
> The frontend relies on these exact shapes. Breaking changes require a spec update first.
|
||||
@@ -1547,7 +1547,7 @@ Theme and branding configuration (merged from config.json + theme.json).
|
||||
```jsonc
|
||||
{
|
||||
"branding": {
|
||||
"siteName": string, // default: "MeshCore Analyzer"
|
||||
"siteName": string, // default: "CoreScope"
|
||||
"tagline": string // default: "Real-time MeshCore LoRa mesh network analyzer"
|
||||
// ... additional branding keys from config/theme files
|
||||
},
|
||||
|
||||
+17
-17
@@ -1,6 +1,6 @@
|
||||
# Migrating from Node.js to Go Engine
|
||||
|
||||
Guide for existing MeshCore Analyzer users switching from the Node.js Docker image to the Go version.
|
||||
Guide for existing CoreScope users switching from the Node.js Docker image to the Go version.
|
||||
|
||||
> **Status (July 2025):** The Go engine is fully functional for production use.
|
||||
> Go images are **not yet published to Docker Hub** — you build locally from source.
|
||||
@@ -24,11 +24,11 @@ Guide for existing MeshCore Analyzer users switching from the Node.js Docker ima
|
||||
## Prerequisites
|
||||
|
||||
- **Docker** 20.10+ and **Docker Compose** v2 (verify: `docker compose version`)
|
||||
- An existing MeshCore Analyzer deployment running the Node.js image
|
||||
- An existing CoreScope deployment running the Node.js image
|
||||
- The repository cloned locally (needed to build the Go image):
|
||||
```bash
|
||||
git clone https://github.com/meshcore-dev/meshcore-analyzer.git
|
||||
cd meshcore-analyzer
|
||||
git clone https://github.com/Kpa-clawbot/meshcore-analyzer.git
|
||||
cd corescope
|
||||
git pull # get latest
|
||||
```
|
||||
- Your `config.json` and `caddy-config/Caddyfile` in place (the same ones you use now)
|
||||
@@ -122,7 +122,7 @@ docker compose --profile staging-go build staging-go
|
||||
Or build directly:
|
||||
|
||||
```bash
|
||||
docker build -f Dockerfile.go -t meshcore-go:latest \
|
||||
docker build -f Dockerfile.go -t corescope-go:latest \
|
||||
--build-arg APP_VERSION=$(git describe --tags 2>/dev/null || echo unknown) \
|
||||
--build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo unknown) \
|
||||
.
|
||||
@@ -151,7 +151,7 @@ Once satisfied, update `docker-compose.yml` to use the Go image for prod:
|
||||
```yaml
|
||||
services:
|
||||
prod:
|
||||
image: meshcore-go:latest # was: meshcore-analyzer:latest
|
||||
image: corescope-go:latest # was: corescope:latest
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.go # add this
|
||||
@@ -174,9 +174,9 @@ docker compose up -d prod
|
||||
./manage.sh stop
|
||||
|
||||
# Build the Go image
|
||||
docker build -f Dockerfile.go -t meshcore-analyzer:latest .
|
||||
docker build -f Dockerfile.go -t corescope:latest .
|
||||
|
||||
# Start (manage.sh uses the meshcore-analyzer:latest image)
|
||||
# Start (manage.sh uses the corescope:latest image)
|
||||
./manage.sh start
|
||||
```
|
||||
|
||||
@@ -248,7 +248,7 @@ These should match (or be close to) your pre-migration numbers.
|
||||
|
||||
```bash
|
||||
# Watch container logs for MQTT messages
|
||||
docker logs -f meshcore-prod --tail 20
|
||||
docker logs -f corescope-prod --tail 20
|
||||
|
||||
# Or use manage.sh
|
||||
./manage.sh mqtt-test
|
||||
@@ -279,13 +279,13 @@ If something goes wrong, switching back is straightforward:
|
||||
```yaml
|
||||
services:
|
||||
prod:
|
||||
image: meshcore-analyzer:latest # back to Node.js
|
||||
image: corescope:latest # back to Node.js
|
||||
# Remove the build.dockerfile line if you added it
|
||||
```
|
||||
|
||||
```bash
|
||||
# Rebuild Node.js image if needed
|
||||
docker build -t meshcore-analyzer:latest .
|
||||
docker build -t corescope:latest .
|
||||
|
||||
docker compose up -d --force-recreate prod
|
||||
```
|
||||
@@ -295,8 +295,8 @@ docker compose up -d --force-recreate prod
|
||||
```bash
|
||||
./manage.sh stop
|
||||
|
||||
# Rebuild Node.js image (overwrites the meshcore-analyzer:latest tag)
|
||||
docker build -t meshcore-analyzer:latest .
|
||||
# Rebuild Node.js image (overwrites the corescope:latest tag)
|
||||
docker build -t corescope:latest .
|
||||
|
||||
./manage.sh start
|
||||
```
|
||||
@@ -310,9 +310,9 @@ docker build -t meshcore-analyzer:latest .
|
||||
Or manually:
|
||||
|
||||
```bash
|
||||
docker stop meshcore-prod
|
||||
docker stop corescope-prod
|
||||
cp backups/pre-go-migration/meshcore.db ~/meshcore-data/meshcore.db
|
||||
docker start meshcore-prod
|
||||
docker start corescope-prod
|
||||
```
|
||||
|
||||
---
|
||||
@@ -348,7 +348,7 @@ docker start meshcore-prod
|
||||
|------|---------|-----|
|
||||
| `engine` field in `/api/health` | Not present or `"node"` | Always `"go"` |
|
||||
| MQTT URL scheme | Uses `mqtt://` / `mqtts://` natively | Auto-converts to `tcp://` / `ssl://` (transparent) |
|
||||
| Process model | Single Node.js process (server + ingestor) | Two binaries: `meshcore-ingestor` + `meshcore-server` (managed by supervisord) |
|
||||
| Process model | Single Node.js process (server + ingestor) | Two binaries: `corescope-ingestor` + `corescope-server` (managed by supervisord) |
|
||||
| Memory management | Configurable via `packetStore.maxMemoryMB` | Loads all packets; no configurable limit |
|
||||
| Startup time | Faster (no compilation) | Slightly slower (loads all packets from DB into memory) |
|
||||
|
||||
@@ -393,4 +393,4 @@ The following gaps have been identified. Check the GitHub issue tracker for curr
|
||||
|
||||
3. **Go ingestor missing `meshcore/self_info` handling** — The local node identity topic is not processed. Low impact but breaks parity.
|
||||
|
||||
4. **No Docker Hub publishing for Go images** — Users must build locally. CI/CD pipeline should publish `meshcore-go:latest` alongside the Node.js image.
|
||||
4. **No Docker Hub publishing for Go images** — Users must build locally. CI/CD pipeline should publish `corescope-go:latest` alongside the Node.js image.
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
#!/bin/bash
|
||||
# MeshCore Analyzer — Setup & Management Helper
|
||||
# CoreScope — Setup & Management Helper
|
||||
# Usage: ./manage.sh [command]
|
||||
#
|
||||
# Idempotent: safe to cancel and re-run at any point.
|
||||
# Each step checks what's already done and skips it.
|
||||
set -e
|
||||
|
||||
CONTAINER_NAME="meshcore-analyzer"
|
||||
IMAGE_NAME="meshcore-analyzer"
|
||||
CONTAINER_NAME="corescope"
|
||||
IMAGE_NAME="corescope"
|
||||
DATA_VOLUME="meshcore-data"
|
||||
CADDY_VOLUME="caddy-data"
|
||||
STATE_FILE=".setup-state"
|
||||
@@ -201,7 +201,7 @@ TOTAL_STEPS=6
|
||||
cmd_setup() {
|
||||
echo ""
|
||||
echo "═══════════════════════════════════════"
|
||||
echo " MeshCore Analyzer Setup"
|
||||
echo " CoreScope Setup"
|
||||
echo "═══════════════════════════════════════"
|
||||
echo ""
|
||||
|
||||
@@ -501,7 +501,7 @@ prepare_staging_config() {
|
||||
if [ ! -f "$staging_config" ] || [ "$prod_config" -nt "$staging_config" ]; then
|
||||
info "Copying production config to staging..."
|
||||
cp "$prod_config" "$staging_config"
|
||||
sed -i 's/"siteName":\s*"[^"]*"/"siteName": "MeshCore Analyzer — STAGING"/' "$staging_config"
|
||||
sed -i 's/"siteName":\s*"[^"]*"/"siteName": "CoreScope — STAGING"/' "$staging_config"
|
||||
log "Staging config created at ${staging_config} with STAGING site name."
|
||||
else
|
||||
log "Staging config is up to date."
|
||||
@@ -541,13 +541,13 @@ cmd_start() {
|
||||
prepare_staging_db
|
||||
prepare_staging_config
|
||||
|
||||
info "Starting production container (meshcore-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
|
||||
info "Starting staging container (meshcore-staging) on port ${STAGING_HTTP_PORT:-81}..."
|
||||
info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
|
||||
info "Starting staging container (corescope-staging) on port ${STAGING_HTTP_PORT:-81}..."
|
||||
docker compose --profile staging up -d
|
||||
log "Production started on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}/${PROD_MQTT_PORT:-1883}"
|
||||
log "Staging started on port ${STAGING_HTTP_PORT:-81} (MQTT: ${STAGING_MQTT_PORT:-1884})"
|
||||
else
|
||||
info "Starting production container (meshcore-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
|
||||
info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
|
||||
docker compose up -d prod
|
||||
log "Production started. Staging NOT running (use --with-staging to start both)."
|
||||
fi
|
||||
@@ -586,12 +586,12 @@ cmd_stop() {
|
||||
if $COMPOSE_MODE; then
|
||||
case "$TARGET" in
|
||||
prod)
|
||||
info "Stopping production container (meshcore-prod)..."
|
||||
info "Stopping production container (corescope-prod)..."
|
||||
docker compose stop prod
|
||||
log "Production stopped."
|
||||
;;
|
||||
staging)
|
||||
info "Stopping staging container (meshcore-staging)..."
|
||||
info "Stopping staging container (corescope-staging)..."
|
||||
docker compose stop staging
|
||||
log "Staging stopped."
|
||||
;;
|
||||
@@ -617,12 +617,12 @@ cmd_restart() {
|
||||
local TARGET="${1:-prod}"
|
||||
case "$TARGET" in
|
||||
prod)
|
||||
info "Restarting production container (meshcore-prod)..."
|
||||
info "Restarting production container (corescope-prod)..."
|
||||
docker compose up -d --force-recreate prod
|
||||
log "Production restarted."
|
||||
;;
|
||||
staging)
|
||||
info "Restarting staging container (meshcore-staging)..."
|
||||
info "Restarting staging container (corescope-staging)..."
|
||||
docker compose --profile staging up -d --force-recreate staging
|
||||
log "Staging restarted."
|
||||
;;
|
||||
@@ -698,19 +698,19 @@ cmd_status() {
|
||||
|
||||
if $COMPOSE_MODE; then
|
||||
echo "═══════════════════════════════════════"
|
||||
echo " MeshCore Analyzer Status (Compose)"
|
||||
echo " CoreScope Status (Compose)"
|
||||
echo "═══════════════════════════════════════"
|
||||
echo ""
|
||||
|
||||
# Production
|
||||
show_container_status "meshcore-prod" "Production"
|
||||
show_container_status "corescope-prod" "Production"
|
||||
echo ""
|
||||
|
||||
# Staging
|
||||
if container_running "meshcore-staging"; then
|
||||
show_container_status "meshcore-staging" "Staging"
|
||||
if container_running "corescope-staging"; then
|
||||
show_container_status "corescope-staging" "Staging"
|
||||
else
|
||||
info "Staging (meshcore-staging): Not running (use --with-staging to start both)"
|
||||
info "Staging (corescope-staging): Not running (use --with-staging to start both)"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -804,7 +804,7 @@ cmd_logs() {
|
||||
docker compose logs -f --tail="$LINES" prod
|
||||
;;
|
||||
staging)
|
||||
if container_running "meshcore-staging"; then
|
||||
if container_running "corescope-staging"; then
|
||||
info "Tailing staging logs..."
|
||||
docker compose logs -f --tail="$LINES" staging
|
||||
else
|
||||
@@ -843,10 +843,10 @@ cmd_promote() {
|
||||
|
||||
# Show what's currently running
|
||||
local staging_image staging_created prod_image prod_created
|
||||
staging_image=$(docker inspect meshcore-staging --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
|
||||
staging_created=$(docker inspect meshcore-staging --format '{{.Created}}' 2>/dev/null || echo "N/A")
|
||||
prod_image=$(docker inspect meshcore-prod --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
|
||||
prod_created=$(docker inspect meshcore-prod --format '{{.Created}}' 2>/dev/null || echo "N/A")
|
||||
staging_image=$(docker inspect corescope-staging --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
|
||||
staging_created=$(docker inspect corescope-staging --format '{{.Created}}' 2>/dev/null || echo "N/A")
|
||||
prod_image=$(docker inspect corescope-prod --format '{{.Config.Image}}' 2>/dev/null || echo "not running")
|
||||
prod_created=$(docker inspect corescope-prod --format '{{.Created}}' 2>/dev/null || echo "N/A")
|
||||
|
||||
echo " Staging: ${staging_image} (created ${staging_created})"
|
||||
echo " Prod: ${prod_image} (created ${prod_created})"
|
||||
@@ -863,8 +863,8 @@ cmd_promote() {
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
if [ -f "$PROD_DATA/meshcore.db" ]; then
|
||||
cp "$PROD_DATA/meshcore.db" "$BACKUP_DIR/"
|
||||
elif container_running "meshcore-prod"; then
|
||||
docker cp meshcore-prod:/app/data/meshcore.db "$BACKUP_DIR/"
|
||||
elif container_running "corescope-prod"; then
|
||||
docker cp corescope-prod:/app/data/meshcore.db "$BACKUP_DIR/"
|
||||
else
|
||||
warn "Could not backup production database."
|
||||
fi
|
||||
@@ -878,7 +878,7 @@ cmd_promote() {
|
||||
info "Waiting for production health check..."
|
||||
local i health
|
||||
for i in $(seq 1 30); do
|
||||
health=$(container_health "meshcore-prod")
|
||||
health=$(container_health "corescope-prod")
|
||||
if [ "$health" = "healthy" ]; then
|
||||
log "Production healthy after ${i}s"
|
||||
break
|
||||
@@ -918,7 +918,7 @@ cmd_update() {
|
||||
|
||||
cmd_backup() {
|
||||
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
|
||||
BACKUP_DIR="${1:-./backups/meshcore-${TIMESTAMP}}"
|
||||
BACKUP_DIR="${1:-./backups/corescope-${TIMESTAMP}}"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
info "Backing up to ${BACKUP_DIR}/"
|
||||
@@ -972,7 +972,7 @@ cmd_restore() {
|
||||
if [ -d "./backups" ]; then
|
||||
echo ""
|
||||
echo " Available backups:"
|
||||
ls -dt ./backups/meshcore-* 2>/dev/null | head -10 | while read d; do
|
||||
ls -dt ./backups/meshcore-* ./backups/corescope-* 2>/dev/null | head -10 | while read d; do
|
||||
if [ -d "$d" ]; then
|
||||
echo " $d/ ($(ls "$d" | wc -l) files)"
|
||||
elif [ -f "$d" ]; then
|
||||
@@ -1019,7 +1019,7 @@ cmd_restore() {
|
||||
|
||||
# Backup current state first
|
||||
info "Backing up current state..."
|
||||
cmd_backup "./backups/meshcore-pre-restore-$(date +%Y%m%d-%H%M%S)"
|
||||
cmd_backup "./backups/corescope-pre-restore-$(date +%Y%m%d-%H%M%S)"
|
||||
|
||||
docker stop "$CONTAINER_NAME" 2>/dev/null || true
|
||||
|
||||
@@ -1105,7 +1105,7 @@ cmd_reset() {
|
||||
|
||||
cmd_help() {
|
||||
echo ""
|
||||
echo "MeshCore Analyzer — Management Script"
|
||||
echo "CoreScope — Management Script"
|
||||
echo ""
|
||||
echo "Usage: ./manage.sh <command>"
|
||||
echo ""
|
||||
|
||||
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "common.proto";
|
||||
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
// ─── Core Channel Type ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
// ─── Pagination ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
+2
-2
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════════════════════
|
||||
// GET /api/config/theme — Theme and branding configuration
|
||||
@@ -10,7 +10,7 @@ option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
|
||||
// Site branding configuration.
|
||||
message Branding {
|
||||
// Site name (default: "MeshCore Analyzer").
|
||||
// Site name (default: "CoreScope").
|
||||
string site_name = 1 [json_name = "siteName"];
|
||||
// Site tagline.
|
||||
string tagline = 2;
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
// ─── Decoded Packet Structure ──────────────────────────────────────────────────
|
||||
// Returned by POST /api/decode, POST /api/packets, and WS broadcast.
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "common.proto";
|
||||
import "packet.proto";
|
||||
|
||||
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "common.proto";
|
||||
import "packet.proto";
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "common.proto";
|
||||
import "decoded.proto";
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "common.proto";
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ syntax = "proto3";
|
||||
|
||||
package meshcore.v1;
|
||||
|
||||
option go_package = "github.com/meshcore-analyzer/proto/v1";
|
||||
option go_package = "github.com/corescope/proto/v1";
|
||||
|
||||
import "decoded.proto";
|
||||
import "packet.proto";
|
||||
|
||||
@@ -344,7 +344,7 @@ app.get('/api/config/theme', (req, res) => {
|
||||
const theme = loadThemeFile();
|
||||
res.json({
|
||||
branding: {
|
||||
siteName: 'MeshCore Analyzer',
|
||||
siteName: 'CoreScope',
|
||||
tagline: 'Real-time MeshCore LoRa mesh network analyzer',
|
||||
...(cfg.branding || {}),
|
||||
...(theme.branding || {})
|
||||
@@ -716,6 +716,10 @@ for (const source of mqttSources) {
|
||||
const role = p.flags ? (p.flags.repeater ? 'repeater' : p.flags.room ? 'room' : p.flags.sensor ? 'sensor' : 'companion') : 'companion';
|
||||
db.upsertNode({ public_key: p.pubKey, name: p.name || null, role, lat: p.lat, lon: p.lon, last_seen: now });
|
||||
if (txResult && txResult.isNew) db.incrementAdvertCount(p.pubKey);
|
||||
// Update telemetry if present in advert
|
||||
if (p.battery_mv != null || p.temperature_c != null) {
|
||||
db.updateNodeTelemetry({ public_key: p.pubKey, battery_mv: p.battery_mv ?? null, temperature_c: p.temperature_c ?? null });
|
||||
}
|
||||
// Invalidate this node's caches on advert
|
||||
cache.invalidate('node:' + p.pubKey);
|
||||
cache.invalidate('health:' + p.pubKey);
|
||||
@@ -1057,6 +1061,10 @@ app.post('/api/packets', requireApiKey, (req, res) => {
|
||||
const role = p.flags ? (p.flags.repeater ? 'repeater' : p.flags.room ? 'room' : p.flags.sensor ? 'sensor' : 'companion') : 'companion';
|
||||
db.upsertNode({ public_key: p.pubKey, name: p.name || null, role, lat: p.lat, lon: p.lon, last_seen: now });
|
||||
if (txResult && txResult.isNew) db.incrementAdvertCount(p.pubKey);
|
||||
// Update telemetry if present in advert
|
||||
if (p.battery_mv != null || p.temperature_c != null) {
|
||||
db.updateNodeTelemetry({ public_key: p.pubKey, battery_mv: p.battery_mv ?? null, temperature_c: p.temperature_c ?? null });
|
||||
}
|
||||
} else {
|
||||
console.warn(`[advert] Skipping corrupted ADVERT (API): ${validation.reason}`);
|
||||
}
|
||||
@@ -2948,7 +2956,7 @@ app.get('/{*splat}', (req, res) => {
|
||||
if (fs.existsSync(indexPath)) {
|
||||
res.sendFile(indexPath);
|
||||
} else {
|
||||
res.status(200).send('<!DOCTYPE html><html><body><h1>MeshCore Analyzer</h1><p>Frontend not yet built.</p></body></html>');
|
||||
res.status(200).send('<!DOCTYPE html><html><body><h1>CoreScope</h1><p>Frontend not yet built.</p></body></html>');
|
||||
}
|
||||
});
|
||||
|
||||
@@ -2959,7 +2967,7 @@ if (require.main === module) {
|
||||
db.removePhantomNodes();
|
||||
server.listen(listenPort, () => {
|
||||
const protocol = isHttps ? 'https' : 'http';
|
||||
console.log(`MeshCore Analyzer running on ${protocol}://localhost:${listenPort}`);
|
||||
console.log(`CoreScope running on ${protocol}://localhost:${listenPort}`);
|
||||
// Log theme file location
|
||||
let themeFound = false;
|
||||
for (const p of THEME_PATHS) {
|
||||
|
||||
@@ -213,6 +213,48 @@ console.log('── Spec Tests: Advert Payload ──');
|
||||
assertEq(p.name, undefined, 'advert no name: name undefined');
|
||||
}
|
||||
|
||||
// Telemetry: sensor node with battery + positive temperature
|
||||
{
|
||||
const pubkey = 'AA'.repeat(32);
|
||||
const sig = 'BB'.repeat(64);
|
||||
const flags = '84'; // sensor(4) | hasName(0x80)
|
||||
const name = Buffer.from('S1').toString('hex') + '00'; // null-terminated
|
||||
const battBuf = Buffer.alloc(2); battBuf.writeUInt16LE(3700);
|
||||
const tempBuf = Buffer.alloc(2); tempBuf.writeInt16LE(2850); // 28.50°C
|
||||
const hex = '1200' + pubkey + '00000000' + sig + flags + name +
|
||||
battBuf.toString('hex') + tempBuf.toString('hex');
|
||||
const p = decodePacket(hex).payload;
|
||||
assertEq(p.battery_mv, 3700, 'telemetry: battery_mv decoded');
|
||||
assert(Math.abs(p.temperature_c - 28.50) < 0.01, 'telemetry: temperature_c positive');
|
||||
}
|
||||
|
||||
// Telemetry: sensor node with 0°C must still emit temperature_c
|
||||
{
|
||||
const pubkey = 'CC'.repeat(32);
|
||||
const sig = 'DD'.repeat(64);
|
||||
const flags = '84'; // sensor(4) | hasName(0x80)
|
||||
const name = Buffer.from('S2').toString('hex') + '00';
|
||||
const battBuf = Buffer.alloc(2); battBuf.writeUInt16LE(3600);
|
||||
const tempBuf = Buffer.alloc(2); // 0°C
|
||||
const hex = '1200' + pubkey + '00000000' + sig + flags + name +
|
||||
battBuf.toString('hex') + tempBuf.toString('hex');
|
||||
const p = decodePacket(hex).payload;
|
||||
assert(p.temperature_c === 0, 'telemetry: 0°C is valid and emitted');
|
||||
}
|
||||
|
||||
// Telemetry: non-sensor node with trailing bytes must NOT decode telemetry
|
||||
{
|
||||
const pubkey = 'EE'.repeat(32);
|
||||
const sig = 'FF'.repeat(64);
|
||||
const flags = '82'; // repeater(2) | hasName(0x80)
|
||||
const name = Buffer.from('R1').toString('hex') + '00';
|
||||
const extraBytes = 'B40ED403'; // battery-like and temp-like bytes
|
||||
const hex = '1200' + pubkey + '00000000' + sig + flags + name + extraBytes;
|
||||
const p = decodePacket(hex).payload;
|
||||
assertEq(p.battery_mv, undefined, 'telemetry: non-sensor node: battery_mv must be undefined');
|
||||
assertEq(p.temperature_c, undefined, 'telemetry: non-sensor node: temperature_c must be undefined');
|
||||
}
|
||||
|
||||
console.log('── Spec Tests: Encrypted Payload Format ──');
|
||||
|
||||
// NOTE: Spec says v1 encrypted payloads have dest(1) + src(1) + MAC(2) + ciphertext
|
||||
|
||||
Reference in New Issue
Block a user