Setting up the devcontainer

The devcontainer file tells GitHub Codespaces how to create your file. You can find my current codespaces file on GitHub.

Bellow, I copied the simplified version that was used while I was writing this post:

{
	"name": "Ubuntu",
	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
	"image": "mcr.microsoft.com/devcontainers/base:jammy",
	"features": {
		"ghcr.io/devcontainers/features/hugo:1": {
			"extended": true
		},
		"ghcr.io/devcontainers-contrib/features/actionlint:1": {},
		"ghcr.io/devcontainers-contrib/features/gh-cli:1": {}
	},
	"customizations": {
		"vscode": {
			"extensions": [
				"eliostruyf.vscode-front-matter"
			]
		}
	},

	// Features to add to the dev container. More info: https://containers.dev/features.
	// "features": {},

	// Use 'forwardPorts' to make a list of ports inside the container available locally.
	"forwardPorts": [1313],

	// You can use go mod instead
	"postCreateCommand": "git submodule update --init",
}

The most important part is forwardPorts, as it will “open” the port that hugo server uses. You can also specify what extensions you want to be installed. I really like FrontMatter, as it has multiple useful features for writing blog posts.

Running hugo

Finally, we have to run hugo, which is, surprisignly, the most difficult part. I just wrote this post to save and publish this command.

We will use some readily available environment variables to set the base url, that is changed by github to allow you and only you to access the open port.

• CODESPACE_NAME: This is the name of the codespace, e.g.: curly-fortnight-asfajkghajhgajhg
• GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN: This is the base domain that github uses for these redirects: app.github.dev

We can combine these two variables to create the URL that will be used by your blog, and enter it into hugo:

hugo server -D --appendPort=false --baseURL https://$CODESPACE_NAME-1313.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN

Ussing FrontMatter’s RUN SERVER button

To finish, the Front Matter extension has a cool “START SERVER” button, we can set what it does by just modifying Visual Studio’s config, and then we will just have to press that button.

The main problem I had with sshfs is that it is not very reliable as a file system to remain mounted 24/7. For example, I had some problems unmounting it when it got stuck in a machine, creating lots of zombie processes. The only solution was to literally restart the machine… Furthermore, its GitHub repo was archived more than a year ago, so these issues will not be addressed soon.

Previous setup

Instead of mounting the volume(s) using fstab, I decided to use a .mount file. It is a systemd unit that you can enable/disable using the systemctl command. You can check if the unit was successfully mounted just using systemctl status. In most systems, a fstab entry will just be converted to a .mount unit anyway. I discovered it recently, but you can just do systemctl cat -- -.mount and see the unit file created to mount your root filesystem.

In my case, the unit file I created was:

[Unit]
Description=Mount publicmedia (/mnt/chikyuu/tdarr/PublicMedia)

[Mount]
# chikyuu hostname defined in /etc/hosts
What=tdarr@chikyuu:PublicMedia
Where=/mnt/chikyuu/tdarr/PublicMedia
Type=fuse.sshfs
Options=noauto,port=222,x-systemd.automount,_netdev,user,default_permissions,allow>
#TimeoutSec=seconds

[Install]
WantedBy=multi-user.target

It’s a lot easier to read, modify and check than using just fstab.

New setup

I configured the NFS server using openmediavault, installed on a Raspberry Pi using Armbian. It was as easy as any other config method. The only difference is that NFS by default doesn’t make any user id translations, but in my case it was fine as I preferred having the original user and groups ids. I had to create the groups on the client machines and change some uids.

You can create a group with a certain group id using the following command:

sudo groupadd -g "gid" "group name"

It’s similar for users, with the command useradd

sudo useradd -u "uid" "user name"

Then, you can add your personal user to a certain group using usermod

sudo usermod -a -G "group name" "user name"

Now, the .mount file is more simple, as you don’t need anything to do with permissions, and you don’t need an IdentityFile. It seems less secure, but I configured the server so only some devices (which are on a separate VLAN) can access the NFS.

[Unit]
Description=Mount publicmedia (/mnt/chikyuu/tdarr/PublicMedia)

[Mount]
# chikyuu hostname defined in /etc/hosts
What=chikyuu:/PublicMedia
Where=/mnt/chikyuu/tdarr/PublicMedia
Type=nfs4
Options=rw,noauto,x-systemd.automount,x-systemd.mount-timeout=10,x-systemd.idle-timeout=1hour

[Install]
WantedBy=multi-user.target

And this is everything, I just restarted the service and everything went smoothly. I still don’t know if it hangs less often, as sshfs hanged just one or two times each month.

Bonus: Mounting using docker-compose

The fact is, I don’t even need the service anymore. I used to mount the filesystem to make it available to some docker containers. But now I can just define it inside the docker-compose.yml file like this:

volumes:
  nfs_publicmedia:
    driver_opts:
      type: nfs4
      o: addr=192.168.1.31,nolock,soft,rw
      device: ':/PublicMedia'
services:
  ubuntu_test:
    image: ubuntu
    volumes:
      - nfs_publicmedia:/media
    command: ls /media && sleep 3600

There’s no need anymore to mount the filesystem in each machine, it is automatically deployed thanks to docker stack!

Important notice: Pocket connection is now officialy available in their integrations page

– https://docs.omnivore.app/using/importing.html#importing-data-from-pocket

Nevertheless, you can read our code if you want to learn about web information processing

I tried to use Pocket, I really tried, but I had to stop using it because it only allowed you 3 highlights per post for free. I know that Pocket is just a company and that they are just trying to provide value to its shareholders, but, come on, is that really a PREMIUM feature? Are you really trying to charge me for a couple of extra bytes?

That’s why I decided to search for another read-it-later app, and I came across Omnivore. It is also Open Source, which would let me self-host the application if I wanted to. I’d be more than happy to pay a fair price for a service I like, but in this case, it isn’t necessary because it’s completely free.

But first, I needed to migrate my data from Pocket to Omnivore. There is currently an open Issue in GitHub (#2050) to provide an official Pocket integration, check in that thread if it has been already implemented before continuing in this tutorial. In my case, after waiting a few weeks, I decided to get on my knees and implement it myself, learning a bit of GraphQL on the way.

Exporting your data from Pocket

This was by far the easiest part, just visit getpocket.com/export and download the ril_export.html file.

Uploading your data to Omnivore

I created a Jupyter Notebook to parse the ril_export.html file and make the proper GraphQL queries to Omnivore’s API. This notebook doesn’t just create the article, it also archives it if it has been read in Pocket, and sets the upload date.

Getting an Omnivore API key

Because we are using a custom-made application, you will need a key to interact with Omnivore. You can find info on how to create an Omnivore API key in the official documentation.

Keep in mind that this API key will be hidden, so make sure to copy it somewhere and don’t lose it, as we will need it in the next step.

If you lose it, don’t worry though, you can always create a new one and delete the unused one.

Running the Jupyter Notebook

I made a Jupyter Notebook using Beautiful Soup to process the ril_export.html file, and gql to run the GraphQL queries, as simple as it is. You can run it online on Binder.

You can also download the notebook from GitHub and run it in VSCode or Jupyter. In that case, install all the packages required by running:

pip install -r https://raw.githubusercontent.com/daviddavo/pocket2omnivore/main/requirements.txt

After running the second cell, It will ask you to upload the ril_export.html file. And then to enter your omnivore API key in the next one. Just paste the API key you obtained in the previous step into the prompt and press ENTER. The text that prompts for the API key can be pretty small, but the notebook won’t continue running until you press ENTER.

If you know any programming, you can also change the OMNIVORE_API_KEY environment variable.

Continuing

After running all the remaining cells, it will start uploading the URLs one by one. It will take a while. If you visit Omnivore, you’ll see that some of these posts don’t have a title or a description. This is because it takes a while to crawl the website.

If it raises a 403 error, the most probable thing is that you entered your API key wrong, or that it expired. Feel free to comment any errors you encounter below or in GitHub Issues.

By the way, did you know that this page is intended to be fully compatible with Omnivore? Try adding some of my posts to your list!

Let’s be honest, if you’re here it’s probably because you found this blog on Google. As I understand it, at the moment there are no comprehensible resources to use ROS2 with the Coppelia simulator (previously known as V-REP), and it is no simple task.

In this tutorial, we will make a simple project using a Pioneer P3DX with a Lidar, in which we will use Slam Toolbox to create a map of the room, all using ROS2 tools like RVIZ2 and NAV2’s Slam Toolbox, but using Coppelia as a simulator instead of Gazebo.

The end product will look something like this:

I made it work but I’m no expert in robotics, so if I made some error, please let me know in the comments or on GitHub.

I will center on the programming part because the way of adding a robot and lidar is the same whether you’re using ROS2 or not.

The original project was made as homework for the Master Universitario en Inteligencia Articial (MUIA) at Universidad Politécnica de Madrid (UPM)

Installing simExtROS2

ROS2 works by creating publishers and subscribers that emit and receive messages of a message type. To be able to do this from Coppelia, we will use the module simExtROS2.

Coppelia comes with this module installed, but it’s compiled with too few message types embedded. If you try publishing some sensor data, it will fail. You need to modify a config file from the source code and compile it again so it supports that message type.

First, we will download the source code. Because I’m using Coppelia 4.4.0, I’ll checkout to that version too. Feel free to change the version number if you need to.

git clone --recursive https://github.com/CoppeliaRobotics/simExtROS2.git
# according to the docs, we have to rename the folder
mv simExtROS2 sim_ros2_interface
git checkout coppeliasim-v4.4.0-rev0

Before compiling, we need to modify a config file to specify the message types you want to compile. This file is meta/interfaces.txt, and in my case, it ended up with the following contents:

builtin_interfaces/msg/Duration
builtin_interfaces/msg/Time
rosgraph_msgs/msg/Clock
geometry_msgs/msg/Quaternion
geometry_msgs/msg/Transform
geometry_msgs/msg/TransformStamped
geometry_msgs/msg/Vector3
geometry_msgs/msg/Point
geometry_msgs/msg/Pose
geometry_msgs/msg/PoseStamped
geometry_msgs/msg/Twist
geometry_msgs/msg/PoseWithCovariance
geometry_msgs/msg/TwistWithCovariance
sensor_msgs/msg/Image
sensor_msgs/msg/PointCloud2
sensor_msgs/msg/PointField
sensor_msgs/msg/LaserScan
std_msgs/msg/Bool
std_msgs/msg/Byte
std_msgs/msg/ColorRGBA
std_msgs/msg/Empty
std_msgs/msg/Float32
std_msgs/msg/Float64
std_msgs/msg/Header
std_msgs/msg/Int8
std_msgs/msg/Int16
std_msgs/msg/Int32
std_msgs/msg/Int64
std_msgs/msg/String
std_msgs/msg/UInt8
std_msgs/msg/UInt16
std_msgs/msg/UInt32
std_msgs/msg/UInt64
std_srvs/srv/Empty
std_srvs/srv/SetBool
std_srvs/srv/Trigger
nav_msgs/msg/Odometry

Ultimately, you will have to add the following environment variable, either modifying ROS’ setup.bash or your .bashrc.

export COPPELIASIM_ROOT_DIR="<your coppelia installation path>"

Now we can proceed to compile and install the library

Compiling libsimExtROS2

After downloading everything, we can install it with Colcon, using the following command:

colcon build --symlink-install

This will take lots of resources and will take up a while the first time, but don’t worry, it will finish and install it.

Modifications in Coppelia

Publishing the simulation time

We can’t use a “wall clock” because the simulation is not in real-time. Let’s imagine you implement an odometry module, and you try to calculate your speed by subtracting your current position from a previous position and dividing by the number of seconds elapsed. This formula will be wrong if your simulation is a bit slow, or if it is too fast. That’s why we need to publish Coppelia’s simulation time into the topic /clock.

function sysCall_init()
  ...
  simTimePub=simROS2.createPublisher('/clock','rosgraph_msgs/msg/Clock')
  ...
end

function sysCall_actuation()
  ...
  t=sim.getSimulationTime()
  simROS2.publish(simTimePub, {clock={
    sec=math.floor(t),
    nanosec=(t-math.floor(t))*10^9}}
  )
  ...
end

function sysCall_cleanup()
  ...
  simROS2.shutdownPublisher(simTimePub)
  ...
end

Publishing transforms

The transforms allow ROS modules to calculate the exact distance between any two objects, publishing just the partial distances. For example, we can publish the distance of the laser’s reference frame to the robot, and ROS can able to calculate the distance from the laser to anything automatically.

Each published transform has a parent, and the root of this hierarchy is the transform that is not published (it is just referenced as a parent). According to the standard REP105, this root should be world or map if we have just one robot.

We will publish the following transforms:

• From the lidar frame to the robot frame
• From the robot to the odometry frame
• From the wheels to the robot frame (needed to display the robot in the simulator)

The final transform, from the odometry frame to the world map is published by another module, so we won’t send it from Coppelia.

To publish all these transforms, we use the following code (remember to change the constants as needed):

function getTransformStamped(objHandle,name,relTo,relToName)
  p=sim.getObjectPosition(objHandle,relTo)
  o=sim.getObjectQuaternion(objHandle,relTo)
  return {
    header={
      stamp=simROS2.getSimulationTime(),
      frame_id=relToName
    },
    child_frame_id=name,
    transform={
      translation={x=p[1],y=p[2],z=p[3]},
      rotation={x=o[1],y=o[2],z=o[3],w=o[4]}
    }
  }
end

function sysCall_actuation()
  ...
  simROS2.sendTransforms({
    getTransformStamped(robotHandle, ROBOT_FRAME_ID, -1, PODOM_FRAME_ID),
    getTransformStamped(leftWheel, 'Pioneer_p3dx_leftWheel', robotHandle, ROBOT_FRAME_ID),
    getTransformStamped(rightWheel, 'Pioneer_p3dx_rightWheel', robotHandle,ROBOT_FRAME_ID),
    getTransformStamped(caster_link, 'Pioneer_p3dx_caster_link', robotHandle, ROBOT_FRAME_ID),
    getTransformStamped(caster_wheel, 'Pioneer_p3dx_caster_wheel', caster_link, 'Pioneer_p3dx_caster_link'),
    getTransformStamped(sensorRefHandle, SENSOR_REF_FRAME, robotHandle, ROBOT_FRAME_ID),
  })
  ...
end

Publishing Lidar PointCloud

The lidar we used returns a PointCloud in Coppelia’s signal Pioneer_p3dx_lidar_data. This PointCloud is an array of triplets of floats, each triplet representing the x, y and z coordinates of each point. We want to publish this data in a topic of type sensor_msgs/msg/PointCloud2.

This message needs us to send a stream of binary data and specify the type of this data in the parameter fields. We will see exactly how with the following commented code:

function sysCall_init()
  ...
  lidarDataTopicName = '/lidarPC'
  lidarDataPub=simROS2.createPublisher(lidarDataTopicName, 'sensor_msgs/msg/PointCloud2')
  ...
end

function sysCall_actuation()
  ...
  lidar_signal = 'Pioneer_p3dx_lidar_data'

  -- Get binary data from signal
  data = sim.getStringSignal(lidar_signal)

  if (data == nil or #data == 0) then
    -- It's normal for this to happend in the first iteration of simulation
    sim.addLog(sim.verbosity_scripterrors, 'signal name ' .. lidar_signal .. ', returned nil value')
  else
    -- Unpack binary data as array of floats
    floats = sim.unpackFloatTable(data)

    -- Each point is 3 floats, so the number of points
    -- is number of floats / 3
    n = #floats//3

    simROS2.publish(lidarDataPub, {
      header={
        stamp=simROS2.getSimulationTime(),
        frame_id='lidar_ref_frame',
      },
      height=1,
      -- Number of points (not bytes)
      width=n,
      -- Each point is 3*4 = 12 bytes
      point_step=12,
      fields={
        -- datatype 7 is FLOAT32
        -- the offset is in BYTES
        {name='x', offset=0, datatype=7, count=1},
        {name='y', offset=4, datatype=7, count=1},
        {name='z', offset=8, datatype=7, count=1},
      },
      -- Unpack data as bytes (uint8)
      data=sim.unpackUInt8Table(data),
    })
  end
  ...
end

After all of this, the PointCloud message will be available on the topic /lidarPC

Setting the robot’s speed

In this case, instead of a publisher, we will need to create a subscriber that receives a message of type Twist. This message contains the desired linear and angular velocities.

We need to convert this to the angular velocity of the motors of the wheels, so, after applying some basic arithmetics, we get the following code:

function sysCall_init()
  ...
  velSub = simROS2.createSubscription('/cmd_vel', 'geometry_msgs/msg/Twist', 'setVelocity_cb')
  ...
end

function setVelocity_cb(msg)
  -- The message provides linear velocity in m/s
  -- but coppelia receives it in rad/s, we need to convert it
  linear = msg.linear.x
  angular = msg.angular.z

  -- https://www.inf.ufrgs.br/~prestes/Courses/Robotics/manual_pioneer.pdf
  wheel_radius = 0.195 / 2 -- 19.5 cmm
  robot_width = 0.33 -- 38 cm minus wheel width
  vl = linear - (angular * robot_width) / 2
  vr = linear + (angular * robot_width) / 2

  sim.setJointTargetVelocity(leftMotor, vl / wheel_radius)
  sim.setJointTargetVelocity(rightMotor, vr / wheel_radius)
end

ROS2

The main advantage of using ROS2 instead of programming the behavior of our robot directly on Coppelia is that ROS2 is platform agnostic and we can use it with any simulator, and even with a real robot.

The system has just 3 nodes:

• pointcloud_to_laserscan: It transforms the data from type Pointcloud2 to LaserScan so Slam Toolbox is able to use it.
• async_toolbox_node: Makes a map and localizes the robot in the map.
• teleop_twist_keyboard: Allows us to move the robot

You’ll need to install them before proceeding

Running the nodes

The easiest way by far is to open three terminals and run the command to start the node in each one. The order doesn’t really matter as nodes usually wait for the info to become available.

Remember to source the ROS setup.bash file to make the ros2 command available!

ros2 run pointcloud_to_laserscan pointcloud_to_laserscan_node --ros-args \
  -p use_sim_time:=true \
  --remap cloud_in:=/lidarPC

ros2 run slam_toolbox async_slam_toolbox_ndoe --ros-args \
  -p base_frame:="base_link" \
  -p odom_frame:="odom" \
  -p map_frame:="map" \
  -p scan_topic:="/scan" \
  -p use_sim_time:=true

ros2 run teleop_twist_keyboard teleop_twist_keyboard

Conclusion

Finally, you can open rviz2 and start adding visualizations to visualize the map that SLAM Toolbox created, you can also visualize the position and velocity of the robot, and even the PointCloud and LaserScan!

Using the teleop_twist_keyboard you can move the robot around, and the map will change in real-time. Pretty fun to drive. If you want, you can try with other nodes, or even create your own to make the robot move autonomously.

If you have any doubts, feel free to leave a comment or open an issue on GitHub. If you already know a bit about robotics and you detect some mistake that I made, please tell me so and I’ll modify the post. Thank you for reading me and see you next time!

Sources and more information

• ArchWiki - ROS. On how to install ROS and solve problems in ArchLinux / Manjaro.
• CoppeliaSim User Manual. One of these things that is not on google, but solves a lot of problems.
• GitHub - coppeliaRobotics/simExtROS2. In the “Issues” part there are some interesting problems and solutions.
• ROS Planning. Setting Up Transformations. NAV2 documentation is overall a good source material to understand how ROS works.
• REP 105 – Coordinate Frames for Mobile Platforms

Since 5 years ago, my CV is written using LaTeX, using (and modifying) the Awesome CV template. On top of that, I added a few things to make it bilingual, compiling either in Spanish or English, making the document available in multiple languages.

Thanks to iflang, depending on the option you pass to the babel package, the document will be rendered in Spanish or English.

\documentclass[a4paper]{awesome-cv}
\usepackage[<lang>]{babel}
\usepackage{iflang}

\newcommand{\es}[1]{\IfLanguageName{spanish}{#1}{}}
\newcommand{\en}[1]{\IfLanguageName{english}{#1}{}}

\begin{document}
  \es{¡Hola Mundo!}
  \en{Hello World!}
\end{document}

Nevertheless, we need to change <lang> by hand, it can’t be automated. A simple option to automate it would be to use sed to modify that file, but it may not be installed in the system (Windows…), and I wanted a pure LaTeX solution.

Juan Carlos Fabero gave me the idea of using a couple of wrappers that define a variable and import the contents of main.tex. We’d have to add a couple of lines to the main file to take that variable into account, checking if it’s been defined.

Modifying main.tex

% Usamos la variable si está definida
\ifdefined\babellang
    \usepackage[\babellang]{babel}
% Si no, usamos el por defecto (o cambiado manualmente desde el IDE)
\else
    \usepackage[english]{babel}
\fi

File english.tex

\def\babellang{english}
\input{main.tex}

File spanish.tex

\def\babellang{spanish}
\input{main.tex}

This way, if we compile english.tex, we will get an English document, and if we compile spanish.tex we’ll get the Spanish one.

Automation with GitHub pages

We can’t forget about doing all this automatically. The source files are located in a GitHub project, and every time a new commit is pushed, the files are compiled and published using GitHub pages. Both files are available through github.io:

• English: https://daviddavo.github.io/22CV/David_Davo_CV_English.pdf
• Spanish: https://daviddavo.github.io/22CV/David_Davo_CV_Spanish.pdf

The workflow used is very simple, I created it just using the web interface, adding a build action that uses latex-action to compile the document, and then a publish action to upload the generated files to GitHub pages. Furthermore, I added an assert to check that the files are only two pages long.

name: CI

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the "main" branch
  push:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - uses: actions/checkout@v3
      - name: Build LaTeX
        uses: xu-cheng/latex-action@v2
        with:
          latexmk_use_xelatex: true
          # Compile multiple documents
          root_file: |
            english.tex
            spanish.tex            
      - name: Assert built files are 2 pages long
        run: |
          sudo apt-get install -y poppler-utils
          [ $(pdfinfo english.pdf | awk '/^Pages:/ {print $2}') -le 2 ]
          [ $(pdfinfo spanish.pdf | awk '/^Pages:/ {print $2}') -le 2 ]          
      - name: Build Github page
        run: |
          mv english.pdf _site/David_Davo_CV_English.pdf
          mv spanish.pdf _site/David_Davo_CV_Spanish.pdf          
      - name: Upload GitHub Pages artifact
        uses: actions/upload-pages-artifact@v1.0.3
  deploy:
    needs: build
    
    # Grant GITHUB_TOKEN the permissions required to make a Pages deployment
    permissions:
      pages: write      # to deploy to Pages
      id-token: write   # to verify the deployment originates from an appropriate source

    # Deploy to the github-pages environment
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}

    # Specify runner + deployment step
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v1
        with:
          artifact-name: build

The final touch is a small raw HTML page that redirects to my personal webpage, in the event someone goes into the root of the page: daviddavo.github.io/22CV

A few days ago, I decided I wanted to refactor my website and create a whole new one, using a static site generator. Ended up deciding on Hugo, btw.

As the old one was a bit, well, clunky, I wanted the new one to be online ASAP as possible. But I needed a banner that said “hey, I’m still working on it, be careful”. Well, I don’t really needed it, but it’s cool nevertheless.

My objective was to emulate a kind of “caution” tape, the yellow and black one that reminds you of roadworks. Making a stripes background wasn’t difficult, thanks to tools like stripesgenerator.com, but the most important thing of all is adding a little tilt, simulating the falling tape.

This could be accomplished thanks to the transform: rotate() CSS property.

It ended up being something like this:

See the Pen Untitled by David Davó (@daviddavo) on CodePen.

What do you think? Pretty neat, huh? If you have any comments on it, share your modifications!.

Issues arise when you use a DNS to assign a domain name to a service. For example, let’s say you have a blog, and you want to map the domain blog.example.com to the public IP 1.2.3.4. In your router, you do a port mapping to the device with a local address 192.168.1.66. But when you want to access blog.example.com in your web browser inside your local network… it doesn’t work! It says something along the lines of “Unable to connect to the host” – but the host is just right THERE! –. The thing is, your computer is trying to connect to 1.2.3.4 (the address resolved by the DNS), when what you really want is to connect to 192.168.1.66 (your local address). You could modify the /etc/hosts file in every device in your home, but there’s an alternative: hosting your own DNS.

Requirements before reading this post

• Knowing what is DNS and how to change the records in your registrar

BIND: The ubiquitous Domain Name System server

With the Domain Name System, you ask a Domain Name server for the IP associated to a Domain Name. I’m pretty sure that you already have a bind system in your home without you knowing it, your ISP given router. In fact, having a DNS server in your LAN has a lot of advantages, because you are querying a device located 10 meters from you, instead of a server hundreds of kilometers from you.

To override some records we can use a Response Policy Zone (RPZ), which allows us to override some domain name mappings inside our local network.

Let’s see how to set up everything

Installing BIND

Depending on your Linux distro or your operating system, the installing process (and the files to modify) may vary. In this tutorial, we use a machine with Debian/Ubuntu. We just need to install a couple of packages, using the following command:

$ sudo apt install bind9 bind9utils bind9-doc

BIND configuration for your local network

Remember: To be able to modify these file you’ll need to be root or use sudo

First, we need to set up a new zone to add our subdomains, we are going to call it “rpz”, and we’ll set its DNS database at /etc/bind/db.rpz. We’ll edit the file /etc/bind/named.conf.default-zones and add the following lines:

zone "rpz" {
    type master;
    file "/etc/bind/db.rpz";
}

Now, we add our records to the database (/etc/bind/db.rpz)

; Our LAN records
$TTL 3600

; These two records specify which is the DNS for this zone
@		IN	SOA	localhost. root.localhost. (
			      2		; Serial
			 604800		; Refresh
			  86400		; Retry
			2419200		; Expire
			 604800 )	; Negative Cache TTL
@		IN	NS	localhost.

; Using $ORIGIN we can add only the subdomain and not the full domain
$ORIGIN example.com

nas                  A       192.168.1.42              ; IPv4 address of nas.example.com
nas                  AAAA    fe80::1ff:fe23:4567:890a  ; IPv6 address of nas.example.com
ftp                  CNAME   nas                       ; Un simple alias of ftp.example.com a nas.example.com
blog.example2.org.   A       192.168.1.31              ; We can also add fully qualified domain names

But we need to tell the DNS to ask for the records (domain name) it doesn’t have in its database to another server. For this, we need to enable recursion and to specify the upstream servers to ask when we don’t have the record in our RPZ.

The forwarders servers can be given by your ISP, or your local router (192.168.1.1), but I recommend using Google’s or Cloudflare’s DNS servers because of their reliability and speed.

options {
	directory "/var/cache/bind";

	recursion yes; # Enabling recursion
	listen-on { 192.168.1.11; }; # The IP address of THIS DNS server
	# You can set it to any if you want it to listen to petitions
	# from outside
	allow-transfer { none; };
	response-policy { zone "rpz"; } break-dnssec yes; # The response policy zone name

	forwarders {
		8.8.8.8;
		8.8.4.4;
		1.1.1.1;
	};

	dnssec-validation auto;

	listen-on-v6 { any; }; # You can set it to "none" if you don't want to use IPv6 DNS
};

It is important to tell the server to ignore dnssec requests. If we don’t do that, the server could reply with a signed response, but with an incorrect (outside) address.

It is important to tell the server to ignore dnssec requests. If we don’t do that, the server could reply with a signed response, but with an incorrect (outside) address.

Starting the DNS server

To apply the changes, we start the server using systemctl

$ sudo systemctl enable --now bind9

We can check whether it works using dig (perhaps you need to install it first), which allows us to make queries to a given DNS server instead of using the default.

First, we check if usual queries work, like google.com

$ dig google.es
; <<>> DiG 9.16.23 <<>> google.es
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5016
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 01db1f47a27adb9e90e4c7fb61c24bc2bf8c6a88296c3923 (good)
;; QUESTION SECTION:
;google.es.			IN	A

;; ANSWER SECTION:
google.es.		300	IN	A	142.250.200.131

;; Query time: 39 msec
;; SERVER: 192.168.1.11#53(192.168.1.11)
;; WHEN: Tue Dec 21 22:48:50 CET 2021
;; MSG SIZE  rcvd: 82

Now we check that our RPZ works

$ dig chikyuu.ddavo.me @192.168.1.11
; <<>> DiG 9.16.23 <<>> nas.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45881
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: d200bc3ba9741e2ab2c8935661c24b4b401d43ae997c39de (good)
;; QUESTION SECTION:
;nas.example.com.		IN	A

;; ANSWER SECTION:
nas.example.com.	5	IN	A	192.168.1.42

;; Query time: 0 msec
;; SERVER: 192.168.1.11#53(192.168.1.11)
;; WHEN: Tue Dec 21 22:46:51 CET 2021
;; MSG SIZE  rcvd: 89

It works!! The returned IP address is the address from our local network, instead of the global one.

Using the DNS server

Nevertheless, it doesn’t work in the web browser, or with any other program apart from dig…

We need to set our DNS server as the default DNS in each device. Every device is different, but it shouldn’t be very difficult. Perhaps your router can propagate its config using DHCP through the local network!

If you have any questions, you can let me know in the comments

Sources & more information

• Serverfault.com: Overriding some DNS entries in BIND for internal networks
• Wikipedia: Response policy zone
• Wikipedia: Network Address Translation
• dnsrpz.info