First of all, thank you for this amazing product. We are working with it now and deploying it on drones and base stations and it’s an excellent way for us to relay information and RTK corrections with each other. Inter-drone communication has never been so easy.
WIth this, comes an issue that we are running into. We like to run a discovery server on each drone, such that they can run without internet. Secondly, we would like would like drones to discover new discovery servers when new drones log into the network.
The issue that we stumbled upon is the requirement that the discovery server list is required at bootup. In your examples on Github discovery servers are hard-coded and not dynamically registered at runtime. Furthermore, it seems that fastDDS examples show that they implement placeholders.. They also mention the existence of ROS_DISCOVERY_SERVER env variable, but is that the way to go?
I was wondering if it is possible that the discovery server list can be dynamically allocated and changed at runtime? For brevity and clarity, what we would like to achieve:
Drones boot up with a local discovery server
They register and monitor the connected devices on Husarnet
Assume that every client on the husarnet network is running a discovery server
Live update their discovery server list such that inter-drone communication is possible.
I was wondering if there is an example implementation of such a situation?
Currently we are considering running the ROS2 Node Client XML file with a only a discoveryServersList pointing to master (Locally on the drone) and then update the ROS_DISCOVERY_SERVER environment variable by reading the /etc/hosts list for changes. Would this work?
Is this the recommended (/necessary) way to go? Would you suggest a different implementation?
Any advice would be highly appreciated.
" … Setting this environment variable to an existing json file allows to load the environment variables from the file instead of from the environment. This allows to change the value of some environment variables at run time …
The latest release of FastDDS is v2.8.x and I can see there a great feature allowing using Husarnet in Discovery Server setups without a custom XML profile file (they have introduced IPv6 support for ROS_DISCOVERY_SERVER env)
Just checked and in the latest release of ROS 2 Humble, there is v2.2.0 version, so to use that feature you will need to install FastDDS from sources.
There is however another interesting option: please take a look at this section about “partitioning using servers”, option no. 3 - Create a new server linked to the servers to which the clients are connected.
So you can introduce “a discovery server for discovery servers”. Thanks to that all your devices will need to have only one, hardcoded address of a “central discovery server” to which all “auxiliary discovery servers” (one for each device) will be connected, and dynamic scaling of your drone fleet should be possible.
I haven’t tested this configuration, but it sounds like something that you are looking for. We have examples for more basic setups with Discovery Server like this one: https://github.com/DominikN/discovery-server-superclient-demo, so maybe it will help a little bit. Please let me know what do you think about that.
A quick question: do you run ROS natively or in Docker?
Dear Dominik,
thank you for the quick reply, highly appreciated.
These options are very interestng. We might opt for option 3, since we’re already running a cloud node for data storage.
However, I also spent some time researching and am opting to go for the DDS-Router implementation. https://husarnet.com/blog/ros2-dds-router I think this fits the use-case. It doesn’t require setting up a master node and we can live update the router’s connection list’s yaml file. As referenced in the github example.
ddsrouter -c /config.yaml -r 10
We are thinking of simply monitoring the /etc/hosts file that husarnet populates and exporting that to this config.yaml file. Although this seems a bit hacky, I think this will work.
We are running every ROS2 node in a docker container, with a husarnet side-car and deploy it through balena. Sadly we are not using the network_mode: service:husarnet as balena only supports network_mode: host/bridge/none, so we are running on host + privileged.