Files
ChameleonUltra/docs/development.md
Philippe Teuwen 8499535aad Clarify protocol. Disruptive changes: see below
This huge commit tries to enhance several things related to the fw/cli protocol.
Generally, the idea is to be verbose, explicit and reuse conventions, in order to enhance code maintainability and understandability for the other contributors.

docs/protocol.md got heavily updated

Many commands have been renamed for consistency. you are invited to adapt your client for easier maintenance

Guidelines, also written in docs/protocol.md "New data payloads: guidelines for developers":
- Now protocol data exchanged over USB or BLE are defined in netdata.h as packed structs and values are stored in Network byte order (=Big Endian)
- Command-specific payloads are defined in their respective cmd_processor handler in app_cmd.c and chameleon_cmd.py
- Define C `struct` for cmd/resp data greater than a single byte, use and abuse of `struct.pack`/`struct.unpack` in Python. So one can understand the payload format at a simple glimpse.
- If single byte of data to return, still use a 1-byte `data`, not `status`.
- Use unambiguous types such as `uint16_t`, not `int` or `enum`. Cast explicitly `int` and `enum` to `uint_t` of proper size
- Use Network byte order for 16b and 32b integers
  - Macros `U16NTOHS`, `U32NTOHL` must be used on reception of a command payload.
  - Macros `U16HTONS`, `U32HTONL` must be used on creation of a response payload.
  - In Python, use the modifier `!` with all `struct.pack`/`struct.unpack`
- Concentrate payload parsing in the handlers, avoid further parsing in their callers. This is true for the firmware and the client.
- In cmd_processor handlers: don't reuse input `length`/`data` parameters for creating the response content
- Avoid hardcoding offsets, use `sizeof()`, `offsetof(struct, field)` in C and `struct.calcsize()` in Python
- Use the exact same command and fields names in firmware and in client, use function names matching the command names for their handlers unless there is a very good reason not to do so. This helps grepping around. Names must start with a letter, not a number, because some languages require it (e.g. `14a_scan` not possible in Python)
- Respect commands order in `m_data_cmd_map`, `data_cmd.h` and `chameleon_cmd.py` definitions
- Even if a command is not yet implemented in firmware or in client but a command number is allocated, add it to `data_cmd.h` and `chameleon_cmd.py` with some `FIXME: to be implemented` comment
- Validate data before using it, both when receiving command data in the firmware and when receiving response data in the client.
- Validate response status in client.

Disruptive changes:
- GET_DEVICE_CAPABILITIES: list of cmds in data are now really Big Endian
  Note: the initial attempt to use macros PP_HTONS were actually considering wrongly that the platform was Big Endian (BYTE_ORDER was actually undefined) while it is actually Little Endian.
- GET_APP_VERSION: response is now a tuple of bytes: major|minor (previously it was in reversed order as a single uint16_t in Little Endian)
- SET_SLOT_TAG_TYPE: tag_type now on 2 bytes, to prepare remapping of its enum
- SET_SLOT_DATA_DEFAULT: tag_type now on 2 bytes, to prepare remapping of its enum
- GET_SLOT_INFO: tag_type now on 2 bytes, to prepare remapping of its enum
- GET_DEVICE_CHIP_ID: now returns its 64b ID following Network byte order (previously, bytes were in the reverse order)
- GET_DEVICE_ADDRESS: now returns its 56b address following Network byte order (previously, bytes were in the reverse order). CLI does not reverse the response anymore so it displays the same value as before.
- MF1_GET_DETECTION_COUNT: now returns its 32b value following Network byte order (previously Little Endian)
- GET_GIT_VERSION response status is now STATUS_DEVICE_SUCCESS
- GET_DEVICE_MODEL response status is now STATUS_DEVICE_SUCCESS
- MF1_READ_EMU_BLOCK_DATA response status is now STATUS_DEVICE_SUCCESS
- GET_DEVICE_CAPABILITIES response status is now STATUS_DEVICE_SUCCESS
- HF14A_SCAN: entirely new response format, room for ATS and multiple tags
- MF1_DETECT_SUPPORT response status is now HF_TAG_OK and support is indicated as bool in 1 byte of data
- MF1_DETECT_PRNG response status is now HF_TAG_OK and prng_type is returned in 1 byte of data with a new enum mf1_prng_type_t == MifareClassicPrngType
- MF1_DETECT_DARKSIDE response status is now HF_TAG_OK and darkside_status is returned in 1 byte of data with a new enum mf1_darkside_status_t == MifareClassicDarksideStatus
- MF1_DARKSIDE_ACQUIRE response status is now HF_TAG_OK and darkside_status is returned in 1 byte of data. If OK, followed by 24 bytes as previously
- MF1_GET_ANTI_COLL_DATA: in case slot does not contain anticoll data, instead of STATUS_PAR_ERR, now it returns STATUS_DEVICE_SUCCESS with empty data
- MF1_SET_ANTI_COLL_DATA and MF1_GET_ANTI_COLL_DATA now use the same data format as HF14A_SCAN

For clients to detect Ultra/Lite with older firmwares, one can issue the GET_APP_VERSION and urge the user to flash his device if needed.
On older firmwares, it will return a status=b'\x00' and data=b'\x00\x01' while up-to-date firmwares will return status=STATUS_DEVICE_SUCCESS and data greater or equal to b'\x01\x00' (v1.0).

Other changes: cf CHANGELOG, and probably a few small changes I forgot about..

TODO:
- remap `tag_specific_type_t` enum to allow future tags (e.g. LF tags) without reshuffling enum and affecting users stored cards
- TEST!
2023-09-18 00:53:39 +02:00

13 KiB

Development

In this file you can look up how to install requirements, edit, compile and debug the firmware!

Prerequisites for compiling

install a cross-compiler

So far, the following compilers have been reported to work fine.
Download one of them and decompress it.
Remember the path where you installed it.

Always use the official versions from ARM, DO NOT install gcc-arm-none-eabi from Debian/Ubuntu.
For some unknown reasons, same gcc version from Debian creates a bootloader too large to fit in the allocated flash space.
Moreover it does not contain the gdb debugger.

install make

  • Debian/Ubuntu alike
    • Open a terminal.
    • Run the following command to install Make: sudo apt-get install build-essential
  • Windows using Chocolatey:
    • Open a PowerShell terminal with administrator privileges.
    • If not yet installed, run the following command to install Chocolatey: Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
    • In the same PowerShell terminal, run the following command to install Make using Chocolatey: choco install make
  • macOS:
    • Open a terminal.
    • If not yet installed, install Homebrew package manager by running the following command: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
    • Once Homebrew is installed, run the following command to install Make: brew install make

install nRF tools

  • Install nRF Util tool nrfutil
    • Move it to a known path like C:\nrfutil\ or /usr/local/bin/
    • Add this path to the PATH Environment Variable if not yet there.
  • Install nRF Util packages:
    • nrfutil install completion device nrf5sdk-tools trace
  • Install nRF Command Line Tools to get nrfjprog, mergehex etc.

install programmer tools

Depending on the hardware programmer you want to use, additional tools are needed.

  • If you are using a J-Link:

    • Install Segger J-Link Software
    • alternatively, you can use openocd as described below
    • Note: a JLink OB (or a STLink reflashed as a JLink OB) will not work on a nRF.
  • If you are using a ST-Link V2:

configure the project

  • Edit Makefile.defs:
    • Change GNU_INSTALL_ROOT (path of previously installed Compiler bin folder)
    • Change GNU_VERSION (Version of the installed Compiler) (FIXME: is it really used?)
    • Change the other paths to match your system if needed
    • Don't forget to remove the # in front of the changed lines
  • Alternatively, if you are committing often code, it may be easier to leave Makefile.defs intact and to invoke make with the desired variables from a script, e.g. make GNU_INSTALL_ROOT=../../../arm-gnu-toolchain-12.2.rel1-x86_64-arm-none-eabi/bin/

Editing the code

You can use Visual Studio Code to edit this project! Simply download and install it!

  • Install the C++ Extension in VS-Code.
  • Install the C++ Extension Pack in VS-Code.
  • Create a new IntelliSense Configuration:
    • press F1 in VS-Code and enter C/C++: Edit Configurations (UI)
    • Add a new Configuration and name it
    • Specify your Compiler path (path of previously installed Compiler bin folder)
    • Change IntelliSense mode to gcc-arm (legacy)
    • Add include path ${workspaceFolder}/**

Compiling the code

  • Install prerequisites (for instructions have a look at Prerequisites for compiling)
  • Run build.sh or try to execute its steps manually if your platform is not yet properly supported. Feedback is always welcome.

The script produces several images in objects.

  • fullimage.hex to be used with a programmer over the SWD pins
  • dfu-app.zip and dfu-full.zip to be used with DFU mode

Uploading the code in DFU mode

If the bootloader and the SoftDevice are already properly installed on the Chameleon, you can reflash it directly over DFU.

To set the device in DFU mode:

  • you can use the Python client and issue the command hw dfu
  • you can use the script resource/tools/enter_dfu.py that does exactly the same but may be easier to call from your scripts
  • you can unplug the device, wait for it to sleep, then press the button B and plug it. If the application is bogus, this is the only way.

The LEDs 4 & 5 should blink green when in DFU mode.

To flash only the application (safer):

nrfutil device program --firmware objects/dfu-app.zip --traits nordicDfu

To flash everything (be sure to also have a JLink or ST-Link V2 programmer if something goes wrong):

nrfutil device program --firmware objects/dfu-full.zip --traits nordicDfu

Under Linux you can use the scripts flash-dfu-app.sh and flash-dfu-full.sh, they will put the device in DFU mode and flash it.

Uploading the code with a programmer

Connect pins GND, SWC (swclk) and SWD (swdio) to your programmer.

With a JLink and nrfjprog

# application only:
nrfjprog -f nrf52 --program objects/application.hex --sectorerase --verify --reset
# full:
nrfjprog -f nrf52 --program objects/fullimage.hex --sectorerase --verify --reset

With a JLink and openocd

# application only:
openocd -f interface/jlink.cfg -f target/nrf52.cfg -c "program objects/application.hex verify reset ; shutdown"
# full:
openocd -f interface/jlink.cfg -f target/nrf52.cfg -c "program objects/fullimage.hex verify reset ; shutdown"

With a ST-Link V2 and openocd

# application only:
openocd -f interface/stlink.cfg -f target/nrf52.cfg -c "program objects/application.hex verify reset ; shutdown"
# full:
openocd -f interface/stlink.cfg -f target/nrf52.cfg -c "program objects/fullimage.hex verify reset ; shutdown"

Uploading the code over BLE

If you are adventurous it is possible to flash the device over BLE (DFU mode).

To put the device in DFU mode

  • you can use the Python client and issue the command hw dfu TODO: this will be possible only when the client will be able to work over BLE...
  • you can use the script resource/tools/enter_dfu_over_ble.py

Once in DFU mode, the device will announce itself over BLE as CU-xxxx where xxxx are the last 2 bytes of the Device Serial Number.

Then use the official nRF Device Firmware Update mobile application to flash one of the DFU images.

Debugging the code from VSCode

  • Install Cortex-Debug VS-Code Extension
  • Open app_main.c
  • Open the extension with CTRL-SHIFT-D
  • Klick on create a launch.json file
  • Select Cortex-Debug
  • Add this in the configuration bracket:
{
    "cwd": "${workspaceFolder}",
    "executable": "${workspaceRoot}/firmware/objects/bootloader.out",
    "name": "Debug with JLink",
    "request": "launch",
    "type": "cortex-debug",
    "runToEntryPoint": "main",
    "showDevDebugOutput": "none",
    "servertype": "jlink",
    "device": "nrf52",
    "interface": "swd",
    "svdFile": "${workspaceRoot}/firmware/nrf52_sdk/modules/nrfx/mdk/nrf52.svd",
}, 
{
    "cwd": "${workspaceFolder}",
    "executable": "${workspaceRoot}/firmware/objects/bootloader.out",
    "name": "Debug with STLink",
    "request": "launch",
    "type": "cortex-debug",
    "runToEntryPoint": "main",
    "showDevDebugOutput": "none",
    "servertype": "openocd",
    "device": "nrf52",
    "svdFile": "${workspaceRoot}/firmware/nrf52_sdk/modules/nrfx/mdk/nrf52.svd",
    "gdbPath": "C:/Program Files (x86)/GNU Arm Embedded Toolchain/10 2021.10/bin/arm-none-eabi-gdb.exe",
    "configFiles": [
        "interface/stlink.cfg",
        "target/nrf52.cfg"
    ]
}
  • If you are jlink probe, create settings.json in {projectRoot}/.vscode directory.
{
    "cortex-debug.armToolchainPath": "C:\\UserProgram\\arm_gcc\\none\\bin",
    "cortex-debug.JLinkGDBServerPath": "C:\\Program Files\\SEGGER\\JLink\\JLinkGDBServerCL.exe",
}
  • To change executable target in launch.json to application or bootloader
  • In the debug menu you can select Debug with JLink or Debug with STLink

Debugging the code with gdb and openocd

See first if you can execute arm-none-eabi-gdb from the installed tools.

  • gcc-arm-none-eabi-10.3-2021.10 gdb requires libncurses5
  • arm-gnu-toolchain-12.2.rel1 gdb requires Python 3.8

In case Python 3.8 is not available anymore on your distro, to install a local copy you can do

wget https://www.python.org/ftp/python/3.8.17/Python-3.8.17.tgz
tar zxvf Python-3.8.17.tgz
cd Python-3.8.17
./configure --prefix=$HOME/opt/python-3.8.17 --enable-shared
make
rm -rf ~/opt/python-3.8.17
make install

Connect openocd to the device with a JLink or a ST-Link V2

openocd -f interface/jlink.cfg -f target/nrf52.cfg
openocd -f interface/stlink.cfg -f target/nrf52.cfg

Then run gdb as follows

PYTHONHOME=~/opt/python-3.8.17/ arm-gnu-toolchain-12.2.rel1-x86_64-arm-none-eabi/bin/arm-none-eabi-gdb

and tell gdb to connect to openocd

target extended-remote localhost:3333

You can reflash a ST-Link V2 to use it as a BlackMagicProbe, to get support for RTT and see NRF_LOG messages. Some clones have only 64kb, this is too short. Even 128kb is too small when enabling RTT, but we can comment parts of the BMP source code.

git clone --recursive git@github.com:blackmagic-debug/stlink-tool.git
( cd stlink-tool && make )

Then put the stlink-tool binary in your path.

Get BMP full sources

Comment out all probes except Nordic nrf51 in src/target/cortexm.c big switch for probes. It should remain

    switch (t->designer_code) {
    case JEP106_MANUFACTURER_NORDIC:
        PROBE(nrf51_probe);
        break;
    }
make -j PROBE_HOST=stlink ST_BOOTLOADER=1 ENABLE_RTT=1

Then flash the ST_Link V2

stlink-tool src/blackmagic.bin

See src/platforms/stlink/README.md for more details. Unplug/plug.
Every time you plug the ST-Link, you have to run stlink-tool to enable BMP.
Under linux, it is convenient to install udev rules to get aliases /dev/ttyBmpGdb and /dev/ttyBmpTarg.

Note that using a native ST-Link V2 with BlackMagicProbe "hosted" will not allow to see NRF_LOG messages.

Debugging the code with gdb and BMP with RTT to monitor NRF_LOG

Assuming you have a BlackMagicProbe with RTT support made out of a ST-Link V2.

RTT usage: https://black-magic.org/usage/rtt.html

stlink-tool
sleep 1
screen /dev/ttyBmpTarg

In another terminal

$ arm-none-eabi-gdb
(gdb) target extended-remote /dev/ttyBmpGdb
(gdb) monitor swdp_scan
 1      Nordic nRF52 M4
 2      Nordic nRF52 Access Port.
(gdb) attach 1
(gdb) monitor rtt

We are now able to use gdb and see the NRF_LOG messages on the other terminal.

cf https://embeddedexplorer.com/nrf52-nrf-log-tutorial/

JLinkExe -if SWD -device nrf52 -speed 4000 -autoconnect 1

in a second terminal:

JLinkRTTClient

Using SWO pin as UART to monitor NRF_LOG

One can set NRF_LOG_UART_ON_SWO_ENABLED := 1 in Makefile.defs to activate this functionality. When activated, NRF_LOG will be available if one connects a UART bridge to the SWO pin which will work as a UART TX pin. UART works at 115200 bauds. E.g. one can use a FTDI dongle and screen /dev/ttyUSB0 115200. Contrary to RTT that needs to be activated by a JTAG probe, UART logs are immediately available.

Limitations:

  • SWO pin is shared with... SWO so when e.g. reflashing the device, garbage may appear on the monitoring terminal.
  • SWO pin is also shared with the blue channel of the RGB slot LEDs, so faint blue may appear briefly when logs are sent and LED might not work properly when supposed to be blue.

Resources