JSON-RPC Beta Testing

[bbernhard]

Hello everybody,

as some of you might have noticed, signal-cli v0.9.0 now supports JSON-RPC. While I wasn't a big fan of signal-cli's DBUS integration (I am just not a big fan of DBUS itself), I really like the JSON-RPC interface. So I figured it would be a good idea to add support for that as well.

With this new mode, the docker image now supports three different modes (the mode can be selected by setting the MODE env variable in the docker-compose.yml appropriately:

• normal mode: every REST API request invokes the signal-cli JAVA application (slowest mode)
• native mode: every REST API request invokes a compiled native image (faster than the normal mode)
• json-rpc mode: the signal-cli JAVA application is started once and the REST API wrapper communicates via JSON-RPC with it (slow startup time, but once the Java application is running, it should be the fastest)

(I haven't tried it yet, but it might be possible to combine the fast startup time of the native image with the JSON-RPC interface. So maybe there will be a fourth mode at some point: json-rpc-native)

How to use it

The only thing you need to do is to set the MODE env variable in the docker-compose.yml file to json-rpc and use the docker image bbernhard/signal-cli-rest-api:0.80-jsonrpc-dev. (I also deprecated some env variables - see the next section for the details)

version: "3"
services:
  signal-cli-rest-api:
    image: bbernhard/signal-cli-rest-api:0.80-jsonrpc-dev
    environment:
      - MODE=json-rpc
      - PORT=8080
    ports:
      - "8080:8080" #map docker port 8080 to host port 8080.
    volumes:
      - "./signal-cli-config:/home/.local/share/signal-cli" #map "signal-cli-config" folder on host system into docker container. the folder contains the password and cryptographic keys when a new number is registered

Limitations, API changes & FAQ:

• Registering, verifying and linking devices only works in normal/native mode. (there's no support yet in signal-cli)
• In JSON-RPC mode it's no longer possible to poll for incoming messages via the HTTP GET receive endpoint. Instead you have to use websockets. e.g: In order to listen for incoming messages you can use wscat like this: wscat -c 127.0.0.1:8080/v1/receive/<your registered signal number> --show-ping-pong --slash
• the USE_NATIVE env variable in the docker-compose.yml file is now deprecated. You can still use it, but you'll see a deprecation warning in the logs. It is recommended to use the MODE env variable instead. (e.g: MODE=native)
• the AUTO_RECEIVE_SCHEDULE env variable in the docker-compose.yml only works in normal/native mode
• the SIGNAL_CLI_CMD_TIMEOUT env variable in the docker-compose.yml only works in normal/native mode
• During testing I've experienced that the first API call is often pretty slow (not sure yet why, but maybe signal-cli is generating some keys?), but the API calls afterwards are really fast.

If anyone wants to have a look at the code, have a look at the jsonrpc git branch.

So far, I've only tested the docker image on x86-64, so not sure if/how much it currently works on arm64/armv7.

Please report any feedback here.

If anyone wants to help with some documentation - any help there is really appreciated! I think with all the different modes and configuration options it's probably becoming more and more difficult for new users to get started. I think the existing README.md could definitely need some love. I am not particularly good at documenting/explaining stuff, so if anyone wants to help me with that, please let me know.

[andreasbrett]

Thanks for this opportunity to get signal-cli REST API back up to speed again on low-power devices. I just powered up the new docker container and right away messages are delivered within 1sec again (normal mode: ~30sec / native mode: was 2-3sec but since an update some time ago also ~20sec).

I will now test the features that I'm using from the REST API but performance wise you just made me very happy! Oh and the websockets I will have to try out for receiving messages.

My device:
CPU = Cortex-A53 (-> arm64)
RAM = 4GB 1866MHz LPDDR3

[andreasbrett]

I can't get the websocket connection to work reliably. Connection drops out every few seconds and apart from events when typing starts and ends I don't get the real message. I tried with the "Websocket In" node in Node Red. Might be an issue with this as I've not used websockets before in Node RED.

What I tested and works though looks good:

• send message to single contact
• send message to group
• create group
• list groups
• list identities

[andreasbrett]

Not sure what I changed but the websockets work now. I can receive messages and they also come in pretty fast. So fast that it's great for implementing Signal Bots through it.

Further successful testing:

• list attachments
• fetch attachment
• remove attachment

[bbernhard]

Many thanks for the feedback - that sounds really good! :)

[f45tb00t]

That's impressive. Sending a message with text only takes now 1second.
But it seems I also needed to remove the " AUTO_RECEIVE_SCHEDULE=0 22 * * *" as the container did not came up otherwise.

[bbernhard]

There's a new version available for testing: bbernhard/signal-cli-rest-api:0.77-jsonrpc-dev

Changelog:

• previously the base image was based on the docker images provided by the adoptopenjdk project. But those images are now deprecated in favor of the eclipse-temurin images. So, we switched to the images provided by eclipse-temurin.
• a few small (mostly cosmetic) changes. (small code improvements, prettier logging messages, etc)

[filipre]

Thanks for the help! I thought it happens because I did not remove the config folder and relink my device. I tried it out and the first time it was fine. When I did a docker-compose down and docker-compose up --force-recreate it then printed the error again.

So I've got empty logs in /var/log/signal-cli-json-rpc-1 and some logs in /var/log/signal-cli-json-rpc-2

root@8da21295fe21:/var/log/signal-cli-json-rpc-2# cat out.log
Invalid username (phone number), make sure you include the country code.
Invalid username (phone number), make sure you include the country code.

I am using the country code as you can see in the jsonrpc2.yml file:

config:
  "+4916901234567":
    tcp_port: 6000
    fifo_pathname: /tmp/sigsocket0
  +rLaaaaaaaaaaaaaH+YaaaaaaaaaaaaaaaaaaaaaY=:
    tcp_port: 6001
    fifo_pathname: /tmp/sigsocket1

The CLI also seems to run since java is being executed. Also receiving messages worked.

    1 signal-+  20   0 1169656  46572  20192 S  71.0   0.4   2:44.71 signal-cli-rest
   39 signal-+  20   0 5707772 172828  23504 S   0.7   1.4   0:29.96 java
   28 root      20   0   31276  19956   5368 S   0.0   0.2   0:00.26 supervisord
   31 signal-+  20   0    6980   3360   3136 S   0.0   0.0   0:00.00 bash
   38 signal-+  20   0    3260    844    752 S   0.0   0.0   0:00.00 nc
  132 signal-+  20   0    6980   3424   3196 S   0.0   0.0   0:00.05 bash
  133 signal-+  20   0    3260    844    752 S   0.0   0.0   0:00.01 nc
  183 root      20   0    7244   4172   3584 S   0.0   0.0   0:00.19 bash
  320 root      20   0    9080   3756   3244 R   0.0   0.0   0:00.03 top

Do I need to change the logging level somewhere?

[bbernhard]

The jsonrpc2.yml file looks a bit strange. Do you have multiple multiple numbers registered?

When the docker container starts in JSON-RPC mode, it checks the signal-cli-config/data folder for all the registered phone numbers. As each registered phone number has its own signal-cli config file inside the signal-cli-config/data folder, all config files with a leading + are assumed to be a registered phone number. So, the jsonrpc2.yml file is basically generated by tree walking the signal-cli-config/data folder. What's a bit strange now is, that the jsonrpc2.yml file contains the entry +rLaaaaaaaaaaaaaH+YaaaaaaaaaaaaaaaaaaaaaY=. That doesn't look like a valid number. And I guess that's the reason why we are seeing the Invalid username (phone number), make sure you include the country code. entries in the log file.

Could you maybe check if you have a file named +rLaaaaaaaaaaaaaH+YaaaaaaaaaaaaaaaaaaaaaY= in your signal-cli-config/data folder? Do you use linked devices? I always assumed that each file in the signal-cli-config/data folder is named like the registered phone number. But maybe that's not true with linked devices? (Haven't used linked devices so far)

[filipre]

I looked at the 0.76 version and I also have that weird entry there. In case it was not obvious, I replaced some characters with a since I don't know if this is some sort of password or unique id or something. When I remove it and restart it gets added automatically. Maybe it failed already in 0.76 but just did not print the error (and thus I did not notice the spam)? Is this line of code new in 0.77 or 0.78?

log.Error("Couldn't read data: ", err.Error())

In fact, when I ssh into the 0.76 container, I see this (see below) so I assume something is not correct either and it just does not print the error. It also explains why I was still able to receive messages, because it read data from the first entry, namely my phone number.

root@d37cdf464b87:/var/log/signal-cli-json-rpc-1# cat out.log
WARN IncomingMessageHandler - Received a group v1 sync message, that can't be handled anymore, ignoring.
WARN IncomingMessageHandler - Received a group v1 sync message, that can't be handled anymore, ignoring.
root@d37cdf464b87:/var/log/signal-cli-json-rpc-1# cd ..
root@d37cdf464b87:/var/log# cd signal-cli-json-rpc-2/
root@d37cdf464b87:/var/log/signal-cli-json-rpc-2# cat out.log
Invalid username (phone number), make sure you include the country code.
root@d37cdf464b87:/var/log/signal-cli-json-rpc-2#

If you want to reproduce this, you probably need to link a device and see, whether you get a second entry after you restart it at least once (it did not show up the first time).

It seems the solution is to drop the assumption that entries there are always phone numbers. But I wonder what it represents?

[bbernhard]

I just linked my Android device, but unfortunately I can't reproduce the issue. All files in my signal-cli-config/data folder are valid phone numbers (also restarted the docker container a bunch of times).

Is this line of code new in 0.77 or 0.78?

According to the git history its there from the beginning.

Is the issue reproducible? i.e If you delete the file (maybe create a backup first) and re-link the device again. Does it happen again?

[filipre]

So I did some more testing and I am also answering your questions that I missed.

Test 1: Container cannot be stopped but must be recreated every time

1. Run docker-compose up --force-recreate signal-cli-rest-api
2. Open http://localhost:8080/v1/about

+ set -e
+ [ -z /home/.local/share/signal-cli ]
+ chown 1000:1000 -R /home/.local/share/signal-cli
+ cat
+ cap_prefix=-cap_
+ cat /proc/sys/kernel/cap_last_cap
+ seq -s ,-cap_ 0 40
+ caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40
+ [ json-rpc = json-rpc ]
+ /usr/bin/jsonrpc2-helper
+ service supervisor start
Starting supervisor: supervisord.
+ supervisorctl start all
+ exec setpriv --reuid=1000 --regid=1000 --init-groups --inh-caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40 signal-cli-rest-api -signal-cli-config=/home/.local/share/signal-cli
time="2021-10-10T13:23:46Z" level=info msg="Started Signal Messenger REST API"
[GIN] 2021/10/10 - 13:26:10 | 404 |        23.4µs |    192.168.32.1 | GET      "/"
[GIN] 2021/10/10 - 13:26:10 | 404 |        22.9µs |    192.168.32.1 | GET      "/favicon.ico"
[GIN] 2021/10/10 - 13:26:16 | 200 |      5.1745ms |    192.168.32.1 | GET      "/v1/about"

3. Stop it docker-compose stop
4. Start it docker-compose start

+ set -e
+ [ -z /home/.local/share/signal-cli ]
+ chown 1000:1000 -R /home/.local/share/signal-cli
+ cat
+ cap_prefix=-cap_
+ cat /proc/sys/kernel/cap_last_cap
+ seq -s ,-cap_ 0 40
+ caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40
+ [ json-rpc = json-rpc ]
+ /usr/bin/jsonrpc2-helper
time="2021-10-10T13:28:24Z" level=fatal msg="Couldn't create fifo with name /tmp/sigsocket0: exit status 1"

Test 2: Removing jsonrpc2.yml just recreates the file

1. docker-compose down and remove jsonrpc2.yml in signal-cli-config volume folder
2. docker-compose up signal-cli-rest-api → recreates jsonrpc2.yml

Creating network "renebot2_default" with the default driver
Creating renebot2_signal-cli-rest-api_1 ... done
Attaching to renebot2_signal-cli-rest-api_1
signal-cli-rest-api_1  | + set -e
signal-cli-rest-api_1  | + [ -z /home/.local/share/signal-cli ]
signal-cli-rest-api_1  | + chown 1000:1000 -R /home/.local/share/signal-cli
signal-cli-rest-api_1  | + cat
signal-cli-rest-api_1  | + cap_prefix=-cap_
signal-cli-rest-api_1  | + cat /proc/sys/kernel/cap_last_cap
signal-cli-rest-api_1  | + seq -s ,-cap_ 0 40
signal-cli-rest-api_1  | + caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40
signal-cli-rest-api_1  | + [ json-rpc = json-rpc ]
signal-cli-rest-api_1  | + /usr/bin/jsonrpc2-helper
signal-cli-rest-api_1  | + service supervisor start
signal-cli-rest-api_1  | Starting supervisor: supervisord.
signal-cli-rest-api_1  | + supervisorctl start all
signal-cli-rest-api_1  | + exec setpriv --reuid=1000 --regid=1000 --init-groups --inh-caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40 signal-cli-rest-api -signal-cli-config=/home/.local/share/signal-cli
signal-cli-rest-api_1  | time="2021-10-10T13:31:13Z" level=info msg="Started Signal Messenger REST API"

Results

• jsonrpc2.yml always gets recreated by signal-cli (correct?)
• You need to recreate the container all the time because stopping leaves the FIFO /tmp/sigsocket0 open and the wrapper tries to recreate it.
• The content of the jsonrpc2.yml always contains my phone number +49123456789 and the weird string id

config:
  "+49*************":
    tcp_port: 6000
    fifo_pathname: /tmp/sigsocket0
  +rL**************H+Y***************************Y=:
    tcp_port: 6001
    fifo_pathname: /tmp/sigsocket1

Questions & Answers

Do you have multiple multiple numbers registered?

I just have one mobile phone number and I am not aware of any other account?

As each registered phone number has its own signal-cli config file inside the signal-cli-config/data folder,

signal-cli-config/data contains one file named +49123456789 and one folder named +49123456789.d but nothing related to the weird string

Could you maybe check if you have a file named +rLaaaaaaaaaaaaaH+YaaaaaaaaaaaaaaaaaaaaaY= in your signal-cli-config/data folder?

Does not exist

According to the git history its there from the beginning.

I still think that the issue only became visible in 0.77 or 0.78. I suppose before it failed earlier but due to the refactoring the error became visible now.

Is the issue reproducible? i.e If you delete the file (maybe create a backup first) and re-link the device again. Does it happen again?

Yes, see tests above.

Next steps

So we have confirmed that the issue happens due to the string not being a phone number. I opened AsamK/signal-cli#772 to understand what the second entry represents. The quick "fix" for me would be to check that config entries only contain numbers and no letters, because then we can assume that it is a valid signal account. But the nicer approach would be to understand, why this entry gets created in the first place for me but not for you.

[bbernhard]

jsonrpc2.yml always gets recreated by signal-cli (correct?)

The jsonrpc2.yml file gets created by the docker container. (see here) and has nothing to do with signal-cli. The jsonrpc2.yml file is basically created by doing a tree walk inside the signal-cli-config/data folder and looking for files starting with +....

edit: But I've just noticed that there was a bug in there. Instead of doing a tree walk in the signal-cli-config/data folder, the whole signal-cli-config directory was traversed recursively. So I guess the mysterious file is probably somewhere in there?

I'll fix that and push a new test version to dockerhub. Hopefully that fixes the issue 🤞

Regarding the "fifo already exists" error: I wasn't able to reproduce the error. But I'll change the code so that it first removes the fifo (in case it already exists) and then creates a new fifo with mkfifo.

[filipre]

Awesome, you were right. It's the first file in the +49123456789.d/group-cache folder storing data about groups, which is exaclty named like the weird string. I'm quite confident that this will fix the issue.

[bbernhard]

A new build is running: https://github.com/bbernhard/signal-cli-rest-api/actions/runs/1326208604

After the github actions job is done, the tag bbernhard/signal-cli-rest-api:0.79-jsonrpc-dev should be on dockerhub.

Changelog:

• fixed directory traversal bug
• improved debug/error logging
• remove fifo before creating a new fifo

[filipre]

Great, making progress! The jsonrpc2.yml looks correct now and I can start and stop the container normally, without recreating it. I just tried out connecting to it using wscat. Though, I think something is still not right with the ping/pong. At the beginning I can still receive messages. However, after a minute or so, the connection drops once I try to receive another message (after I send a message in signal to a group). Take a look at my logs

wscat

(base) ~> wscat -c 127.0.0.1:8080/v1/receive/+**************** --show-ping-pong --slash
Connected (press CTRL+C to quit)
< {"envelope":{"source":"+****************","sourceNumber":"+****************","sourceUuid":"****************","sourceName":"****************","sourceDevice":1,"timestamp":1633893434401,"syncMessage":{"sentMessage":{"timestamp":1633893434401,"message":"Test","expiresInSeconds":0,"viewOnce":false,"mentions":[],"attachments":[],"contacts":[],"groupInfo":{"groupId":"****************=","type":"DELIVER"},"destination":null,"destinationNumber":null,"destinationUuid":null}}}}
< Received ping
< {"envelope":{"source":"+****************","sourceNumber":"+****************","sourceUuid":"****************","sourceName":"****************","sourceDevice":1,"timestamp":1633893486715,"syncMessage":{"sentMessage":{"timestamp":1633893486715,"message":"Test","expiresInSeconds":0,"viewOnce":false,"mentions":[],"attachments":[],"contacts":[],"groupInfo":{"groupId":"****************","type":"DELIVER"},"destination":null,"destinationNumber":null,"destinationUuid":null}}}}
< Received ping
< Received ping
< Received ping
< Received ping
< Received ping

wrapper

!master ~/Development/signal-bot/renebot2> docker-compose up signal-cli-rest-api
Starting renebot2_signal-cli-rest-api_1 ... done
Attaching to renebot2_signal-cli-rest-api_1
signal-cli-rest-api_1  | + set -e
signal-cli-rest-api_1  | + [ -z /home/.local/share/signal-cli ]
signal-cli-rest-api_1  | + chown 1000:1000 -R /home/.local/share/signal-cli
signal-cli-rest-api_1  | + cat
signal-cli-rest-api_1  | + cap_prefix=-cap_
signal-cli-rest-api_1  | + cat /proc/sys/kernel/cap_last_cap
signal-cli-rest-api_1  | + seq -s ,-cap_ 0 40
signal-cli-rest-api_1  | + caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40
signal-cli-rest-api_1  | + [ json-rpc = json-rpc ]
signal-cli-rest-api_1  | + /usr/bin/jsonrpc2-helper
signal-cli-rest-api_1  | time="2021-10-10T19:16:57Z" level=info msg="Found number +********** and added it to jsonrpc2.yml"
signal-cli-rest-api_1  | + service supervisor start
signal-cli-rest-api_1  | Starting supervisor: supervisord.
signal-cli-rest-api_1  | + supervisorctl start all
signal-cli-rest-api_1  | + exec setpriv --reuid=1000 --regid=1000 --init-groups --inh-caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40 signal-cli-rest-api -signal-cli-config=/home/.local/share/signal-cli
signal-cli-rest-api_1  | time="2021-10-10T19:16:58Z" level=info msg="Started Signal Messenger REST API"
signal-cli-rest-api_1  | time="2021-10-10T19:22:47Z" level=error msg="Couldn't write message: write tcp 172.20.0.2:8080->172.20.0.1:65078: i/o timeout"
signal-cli-rest-api_1  | [GIN] 2021/10/10 - 19:23:33 | 200 |         6m24s |      172.20.0.1 | GET      "/v1/receive/+**********"

I'd have expected that the client stays connected for hours and days, just as it was in 0.76. The connection only drops once I send a new message after some time. Do I need to configure different ping intervals in the client? I tried out 5s which did not work for me.

[bbernhard]

Many thanks for letting me know! I can reproduce the issue here too. I'll have a look.

[bbernhard]

New build is running: https://github.com/bbernhard/signal-cli-rest-api/actions/runs/1326608673

After the github actions job is done, the tag bbernhard/signal-cli-rest-api:0.80-jsonrpc-dev should be on dockerhub.

I've looked again at the websocket rfc and the websocket rfc doesn't specify what to do in case a client doesn't respond to ping. It just states, that a websocket client must respond with a pong, but it doesn't specify any consequences when that's not the case. The websocket library I am using (gorilla websocket) has a built in functionality to raise an error in case the read/write deadlines aren't met.

I've googled a bit and found a few websocket server implementations which also ignore a missing pong. So I basically did the same. Hope it works now :)

[filipre]

Thanks again. I tested it quickly and it looks good. I'll run a longer test over the weekend.

[filipre]

App is running for >3 days. No issues detected so far 🎉

[bbernhard]

Thanks a lot for the feedback @filipre!

I guess we should be good to go for a first release then 🎉

I'll push the first stable version to dockerhub in the next days (still need to write a bit of documentation for the JSON-RPC mode).

Many thanks to @andreasbrett @filipre @dattito @f45tb00t for all the testing and helpful feedback!

[haberda]

@bbernhard When I set the mode to json-rpc it seems to want to look here: /home/.local/share/signal-cli/data causing a fatal error, rather than the location I specify with the environment variable. I suspect it is hard coded somewhere to look for that DIR rather than using the variable. Native and normal modes work fine with the alternative location.

Specific error message:

level=fatal msg="Couldn't read contents of /home/.local/share/signal-cli/data. Is your phone number properly registered? Please be aware that registering a phone number only works in normal/native mode and is currently not supported in json-rpc mode!

When:

SIGNAL_CLI_CONFIG_DIR=/data

[bbernhard]

You are right, completely forgot about that.

Should be fixed now. There's a new build running which contains the fix. After the build is finished, the image with the tag bbernhard/signal-cli-rest-api:0.81-signal-cli-config-dir-fix-dev should be available on Dockerhub for testing.

Please let me know if everything works now. I'll then release a new stable image.

[haberda]

Ok, now it seems like it's going to the right place but I get this error:

level=error msg="Couldn't read data for number <number>: EOF. Is the number properly registered?"

The number works fine in native or normal modes. I haven't tried to delete and re-register which may help.

[bbernhard]

Are you maybe overwriting the entrypoint in your addon?

The JSON-RPC mode is a little bit more complicated and requires some orchestration in the background. i.e: In JSON-RPC mode, a signal-cli instance is started in the background (controlled by supervisord) and the REST API communicates with the signal-cli instance via TCP. On a first glance it looks like that supervisorctl either doesn't spawn the signal-cli instance or it somehow crashes.

Do you see something like that in the docker-compose output?

signal-cli-rest-api_1  | + service supervisor start
signal-cli-rest-api_1  | Starting supervisor: supervisord.
signal-cli-rest-api_1  | + supervisorctl start all

Could you please connect to the running docker container with docker exec -it <container id> /bin/bash and check the /var/log/signal-cli-json-rpc-1/ folder for log files. Do you see any error in there? Please also check the log file /var/log/supervisor/supervisord.log.

[haberda]

Sorry about the delay, I've been traveling. Anyway, here is the initial log output before the error before spams the console:

+ set -e
+ [ -z /data ]
+ chown 1000:1000 -R /data
+ cat
+ cap_prefix=-cap_
+ cat /proc/sys/kernel/cap_last_cap
+ seq -s ,-cap_ 0 40
+ caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40
+ [ json-rpc = json-rpc ]
+ /usr/bin/jsonrpc2-helper
time="2021-11-06T14:29:31-07:00" level=info msg="Found number <mynumber> and added it to jsonrpc2.yml"
+ service supervisor start
Starting supervisor: supervisord.
+ supervisorctl start all
+ exec setpriv --reuid=1000 --regid=1000 --init-groups --inh-caps=-cap_0,-cap_1,-cap_2,-cap_3,-cap_4,-cap_5,-cap_6,-cap_7,-cap_8,-cap_9,-cap_10,-cap_11,-cap_12,-cap_13,-cap_14,-cap_15,-cap_16,-cap_17,-cap_18,-cap_19,-cap_20,-cap_21,-cap_22,-cap_23,-cap_24,-cap_25,-cap_26,-cap_27,-cap_28,-cap_29,-cap_30,-cap_31,-cap_32,-cap_33,-cap_34,-cap_35,-cap_36,-cap_37,-cap_38,-cap_39,-cap_40 signal-cli-rest-api -signal-cli-config=/data
time="2021-11-06T14:29:39-07:00" level=info msg="Started Signal Messenger REST API"

The log file in /var/log/signal-cli-json-rpc-1/ simply says:

User <mynumber> is not registered.

Here is /var/log/supervisor/supervisord.log:

2021-11-06 21:29:36,749 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in
 the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-11-06 21:29:36,749 WARN For [program:signal-cli-json-rpc-1], redirect_stderr=true but stderr_logfile has also been
set to a filename, the filename has been ignored
2021-11-06 21:29:36,749 WARN For [program:signal-cli-json-rpc-2], redirect_stderr=true but stderr_logfile has also been
set to a filename, the filename has been ignored
2021-11-06 21:29:36,749 INFO Included extra file "/etc/supervisor/conf.d/signal-cli-json-rpc-1.conf" during parsing
2021-11-06 21:29:36,750 INFO Included extra file "/etc/supervisor/conf.d/signal-cli-json-rpc-2.conf" during parsing
2021-11-06 21:29:36,806 INFO RPC interface 'supervisor' initialized
2021-11-06 21:29:36,806 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-11-06 21:29:36,809 INFO daemonizing the supervisord process
2021-11-06 21:29:36,811 INFO supervisord started with pid 34
2021-11-06 21:29:37,816 INFO spawned: 'signal-cli-json-rpc-1' with pid 36
2021-11-06 21:29:37,820 INFO spawned: 'signal-cli-json-rpc-2' with pid 37
2021-11-06 21:29:38,822 INFO success: signal-cli-json-rpc-1 entered RUNNING state, process has stayed up for > than 1 se
conds (startsecs)
2021-11-06 21:29:38,822 INFO success: signal-cli-json-rpc-2 entered RUNNING state, process has stayed up for > than 1 se
conds (startsecs)
2021-11-06 21:29:44,773 INFO exited: signal-cli-json-rpc-1 (exit status 1; not expected)
2021-11-06 21:29:45,775 INFO spawned: 'signal-cli-json-rpc-1' with pid 142
2021-11-06 21:29:46,776 INFO success: signal-cli-json-rpc-1 entered RUNNING state, process has stayed up for > than 1 se
conds (startsecs)
2021-11-06 21:29:48,023 INFO exited: signal-cli-json-rpc-2 (exit status 1; not expected)
2021-11-06 21:29:49,025 INFO spawned: 'signal-cli-json-rpc-2' with pid 197
2021-11-06 21:29:50,025 INFO success: signal-cli-json-rpc-2 entered RUNNING state, process has stayed up for > than 1 se
conds (startsecs)

Switching back to native or normal modes works fine. I still haven't tried to register again, it might help.

[bbernhard]

I think I've found the issue. Another build is already running. After the build is finished, the image with the tag bbernhard/signal-cli-rest-api:0.82-signal-cli-config-dir-fix-dev should be available on Dockerhub for testing.

[haberda]

Yep, works now! Thanks again!

[bbernhard]

You're welcome, glad it worked! I'll release a stable version in the next days :)