Installing Gentoo on a Virtual Private Server

Unknown — Tue, 20 Feb 2024 00:00:00 +0000

I’ve been running Gentoo on my servers for ages. Recently I moved from a hosted hardware server to a set of virtual ones, running at OVH Cloud</a> and Oracle Cloud</a>.

Below is an example of how I setup 2 servers with Gentoo on Oracle Cloud while gentoo is not offered. This is done in 3 main steps:

Installing Gentoo on a local VM on our own environment</li>

Pushing this VM image onto the VPS</li>

Finishing the setup and enjoying Gentoo</li> </ol>
This is mostly written as a reminder on how to do it and it may voluntarily skip many details.
On installing Gentoo itself, I suggest reading the Gentoo Handbook</a>.
Local Gentoo Install</h1>
On the home computer, I download the gentoo iso from links on gentoo.org</a> and verify it:
wget https://distfiles.gentoo.org/releases/amd64/autobuilds/20240218T170410Z/install-amd64-minimal-20240218T170410Z.iso{,.asc} gpg --import /usr/share/openpgp-keys/gentoo-release.asc gpg --verify install-amd64-minimal-*.iso.asc </code></pre> Then, I spawn an instance on OCI with RockyLinux and configure ssh by adding to the .ssh/config</code> on our local machine:
Host vpsoci Hostname IP_ADDR_HERE User opc IdentityFile ~/.ssh/id_ed25519 </code></pre> Test it by downloading dmesg</code> and lsmod</code> to keep as reference for later:
ssh vpsoci "dmesg" > dmesg ssh vpsoci "lsmod" > lsmod </code></pre> Get the vps size:
ssh vpsoci "sudo sgdisk -p /dev/sda | head -n 1" </code></pre> that outputs Disk /dev/sda: 97677312 sectors, 46.6 GiB</code>. Next I create our own disk image and start our own local VM and connect to it:
qemu-img create -f raw gentoo.raw $((97677312 * 512)) qemu-system-x86_64 -drive file=gentoo.raw,format=raw,cache=writeback -cdrom install-amd64-minimal-*.iso -boot d -machine type=q35,accel=kvm -cpu host -smp 2 -m 1024 -vnc :0 -device virtio-net,netdev=vmnic -netdev user,id=vmnic,hostfwd=tcp::10022-:22 vncviewer :0 </code></pre> On it, I add a password and start ssh:
passwd rc-service sshd start </code></pre> Now I can ssh root@localhost -p 10022</code> and partition the VM:sgdisk -n1:0:+2M -t1:EF02 /dev/sda sgdisk -n2:0:+512M -t2:EF00 /dev/sda sgdisk -n3:0:0 -t3:8300 /dev/sda mkfs.fat -F 32 -n efi-boot /dev/sda2 </code></pre> And I setup dm-crypt on the whole disk. I kept the default settings. To pick a password, I suggesting following Randall Munroe’s advice at xkcd</a>. cryptsetup luksFormat /dev/sda3 cryptsetup luksDump /dev/sda3 cryptsetup luksOpen --allow-discards /dev/sda3 root mkfs.ext4 -m 1 /dev/mapper/root mount /dev/mapper/root /mnt/gentoo </code></pre> Then, I install the base system on it cd /mnt/gentoo wget https://distfiles.gentoo.org/releases/amd64/autobuilds/20240218T170410Z/stage3-amd64-openrc-20240218T170410Z.tar.xz{,.asc} gpg --import /usr/share/openpgp-keys/gentoo-release.asc gpg --verify stage3-*.tar.xz.DIGESTS tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner rm stage3-*.tar.* </code></pre> And edit the portage configuration: vi /mnt/gentoo/etc/portage/make.conf mkdir --parents /mnt/gentoo/etc/portage/repos.conf cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf </code></pre> Prepare to chroot cp --dereference /etc/resolv.conf /mnt/gentoo/etc/ mount --types proc /proc /mnt/gentoo/proc mount --rbind /sys /mnt/gentoo/sys && mount --make-rslave /mnt/gentoo/sys mount --rbind /dev /mnt/gentoo/dev && mount --make-rslave /mnt/gentoo/dev mount --bind /run /mnt/gentoo/run && mount --make-slave /mnt/gentoo/run chroot /mnt/gentoo /bin/bash source /etc/profile export PS1="(chroot) ${PS1}" </code></pre> Once in the chroot, I update and start configuring the system: emerge-webrsync eselect profile set 15 echo "Europe/Paris" > /etc/timezone rm /etc/localtime emerge --config sys-libs/timezone-data cat <<EOF > /etc/locale.gen en_DK.UTF-8 UTF-8 en_US.UTF-8 UTF-8 fr_FR.UTF-8 UTF-8 C.UTF8 UTF-8 EOF locale-gen eselect locale list eselect locale set 4 env-update && source /etc/profile && export PS1="(chroot) ${PS1}" </code></pre> Install gentoo gpg keys: emerge -qaAv getuto && getuto </code></pre> Add -v3</code> to the sync-uri</code> of /etc/portage/binrepos.conf/gentoobinhost.conf</code> to be able to install binary packages more fit for our target system. Install base components: emerge -qaAvg sys-apps/fakeroot app-admin/sysklogd sys-process/cronie net-misc/chrony app-editors/neovim rc-update add sysklogd default rc-update add cronie default rc-update add sshd default rc-update add chronyd default </code></pre> The kernel</h2> I like configuring my kernels and use genkernel</code> to build them. Here it is setup to allow an ssh connection to the initramfs to be able to decrypt the root volume. echo "sys-kernel/genkernel -firmware" >> /etc/portage/package.use/main echo "sys-kernel/gentoo-sources experimental" >> /etc/portage/package.use/main net-misc/dropbear static -pam sys-libs/zlib static-libs virtual/libcrypt static-libs sys-libs/libxcrypt static-libs sys-apps/kmod zstd static-libs emerge -qaAvg sys-kernel/gentoo-sources sys-kernel/genkernel net-misc/dropbear </code></pre> I edit /etc/dropbear/authorized_keys</code> with the keys I want to use. Set GRUB_PLATFORMS="efi-64 emu pc"</code> to /etc/portage/make.conf</code>. Set GRUB_CMDLINE_LINUX_DEFAULT="root_trim=yes crypt_root=/dev/sda3 ip=dhcp dosshd gk.sshd.port=2222"</code> to /etc/default/grub</code>. Using port 2222 helps make ssh not complain that the host is different from the initramfs and the live system. Finish grub setup: mount /dev/sda2 /boot echo "sys-boot/grub device-mapper efiemu" >> /etc/portage/package.use/main emerge -qaAvg sys-boot/grub grub-install --target=x86_64-efi --efi-directory=/boot --removable grub-mkconfig -o /boot/grub/grub.cfg </code></pre> Generate the kernel configuration and then run: genkernel all --kernel-config=/usr/src/kernel-config-6.1.67-v1 </code></pre> Configuring the system</h2> Run blkid</code> to get the UUID of /dev/mapper/root</code> and then write /etc/fstab</code> like so: # <fs> <mountpoint> <type> <opts> <dump> <pass> LABEL=efi-boot /boot vfat noauto,noatime 0 2 #/dev/mapper/root UUID="1ec95162-17a6-462c-b0ba-00878f74fe29" / ext4 noatime 0 1 </code></pre> Set the hostname. Please have more inspiration. echo vpsoci > /etc/hostname </code></pre> For the network, just use dhcpcd: emerge -qaAdg dhcpcd rc-update add dhcpcd default rc-service dhcpcd start </code></pre> Add a udev rule to rename enp0s3</code> to eth0</code>: rc-config add udev mkdir -p /etc/udev/rules.d/ cat <<EOF > /etc/udev/rules.d/76-net-name-use-custom.rules SUBSYSTEM=="net", ACTION=="add", ENV{ID_NET_NAME_PATH}=="enp0s3", NAME="eth0" EOF </code></pre> Copying the image to the VPS</h1> Here’s the trick given by Adyxax when he installed NixOS on a VPS</a>: the idea is to rewrite the disk image on the vps with the one from the local VM. This can be done by writing directly to the block device, bypassing the filesystem that has to be remounted read-only to avoid it writing over our data. On the VPS, as root, after ensuring root can connect directly (need to modify /root/.ssh/authorized_keys</code>), this can be done like so: dnf install zstd swapoff -a mount -o remount,ro /boot mount -o remount,ro /boot/efi systemctl stop systemd-journald.service systemd-journald-dev-log.socket systemd-journald.socket systemctl stop rsyslog.service crond.service chronyd.service gssproxy.service systemctl stop systemd-udevd.service systemd-udevd-control.socket systemd-udevd-kernel.socket systemctl stop rpcbind.service rpcbind.socket systemctl stop tuned.service polkit.service atd.service auditd.service libstoragemgmt.service systemctl stop tuned.service polkit.service atd.service libstoragemgmt.service systemctl stop oracle-cloud-agent-updater.service oracle-cloud-agent.service pmcd.service pmie.service pmie_farm.service systemctl stop firewalld.service irqbalance.service pmlogger.service pmlogger_farm.service systemctl stop dtprobed.service systemctl stop getty@tty1.service serial-getty@ttyS0.service service auditd stop mount -o remount,ro /var/oled systemctl stop dbus-broker.service systemd-logind.service dbus.socket systemctl stop iscsid.service iscsid.socket mount -o remount,ro / </code></pre> And the long command to run from your environment: zstd -k -3 gentoo.raw --stdout | ssh -4 root@vpsoci "zstdcat | dd of=/dev/sda"; echo -e \a </code></pre> Finishing the install</h1> Reboot the VPS, and hopefully enjoy Gentoo. Note on rewriting the root volume</h2> This may no longer be possible directly in the future starting with kernel 6.8 when the option BLK_DEV_WRITE_MOUNTED</code> will be set to no: Allow writing to mounted block devices When a block device is mounted, writing to its buffer cache is very likely going to cause filesystem corruption. It is also rather easy to crash the kernel in this way since the filesystem has no practical way of detecting these writes to buffer cache and verifying its metadata integrity. However there are some setups that need this capability like running fsck on read-only mounted root device, modifying some features on mounted ext4 filesystem, and similar. If you say N, the kernel will prevent processes from writing to block devices that are mounted by filesystems which provides some more protection from runaway privileged processes and generally makes it much harder to crash filesystem drivers. Note however that this does not prevent underlying device(s) from being modified by other means, e.g. by directly submitting SCSI commands or through access to lower layers of storage stack. If in doubt, say Y. The configuration can be overridden with the bdev_allow_write_mounted boot option. </blockquote> See also this great article from LWN: Defending mounted filesystems from the root user</a>

Coccinelle

Unknown — Fri, 01 Jul 2022 00:00:00 +0000

Coccinelle (meaning ladybug in French) can be a gardener’s good friend and even though I enjoy taking care of my house plants, I am writing here a bit about software development.

Being a C developer, I know how dangerous the language can be and how important it is to be able to protect oneself when using it.

This is the reason why Coverity</a> is used on Terminology. It does provide some static analysis on Terminology’s code.

However, it may not be enough or not have all the rules I wish.

What is Coccinelle?</h1>

Coccinelle is a program matching and transformation engine which provides the language SmPL (Semantic Patch Language) for specifying desired matches and transformations in C code. </blockquote>

Coccinelle can read C code and allow to write semantic patches on it. Such patches can be used to use a new API (or macro) like in the following example:

@@
</span>expression n,d;
</span>@@
</span>
</span>(
</span>-</span> (((n + d) - 1) / d)
</span></span>+</span> DIV_ROUND_UP(n,d)
</span></span>|
</span>-</span> ((n + (d - 1)) / d)
</span></span>+</span> DIV_ROUND_UP(n,d)
</span></span>)
</span></code></pre>
This script will generate patches to use the macro DIV_ROUND_UP()</code> to make
the code easier to read.</p>
Longer example</h1>
Recently, I stumbled accross some code that was messing up with the type of
enum a function should return.  Below is an example of bad code:</p>
enum</span> foo</span></span></span> {</span></span></span>
</span></span></span>    FOO_1,</span>
</span></span></span>    FOO_2,</span>
</span></span></span>    FOO_3
</span></span></span></span></span></span>}</span></span></span>;</span>
</span>
</span>enum</span> bar</span></span></span> {</span></span></span>
</span></span></span>    BAR_1,</span>
</span></span></span>    BAR_2
</span></span></span></span></span></span>}</span></span></span>;</span>
</span>
</span>enum</span> bar</span></span></span>
</span></span>my_func(void)
</span></span>{</span></span></span>
</span></span></span>    enum</span> foo f =</span> FOO_3;</span>
</span></span></span>
</span></span></span>    do_something</span>(</span></span></span></span>)</span></span></span>;</span>
</span></span></span>
</span></span></span>    return</span> f;</span>
</span></span></span></span></span>}</span></span></span>
</span></code></pre>
Clang and Coverity can find such implicit enum conversions but I wanted to see
if I could write some Coccinelle patch to find and fix such cases.</p>
Here’s what I wrote:</p>
1</td>     @@
</span></td></tr>
2</td>     type T, B != T;
</span></td></tr>
3</td>     identifier func, i;
</span></td></tr>
4</td>     expression E;
</span></td></tr>
5</td>     @@
</span></td></tr>
6</td> 
</span></td></tr>
7</td>     T func (...) {
</span></td></tr>
8</td>     ...
</span></td></tr>
9</td>     (
</span></td></tr>
10</td>     T i;
</span></td></tr>
11</td>     |
</span></td></tr>
12</td>     T i = E;
</span></td></tr>
13</td>     |
</span></td></tr>
14</td>     -B i;
</span></td></tr>
15</td>     +T i;
</span></td></tr>
16</td>     |
</span></td></tr>
17</td>     -B i = E;
</span></td></tr>
18</td>     +T i = E;
</span></td></tr>
19</td>     )
</span></td></tr>
20</td>     <+...
</span></td></tr>
21</td>     return i;
</span></td></tr>
22</td>     ...+>
</span></td></tr>
23</td>     }
</span></td></tr></tbody></table></code></pre>
On line 7, the return type of the function is set to T</code>.</p>
Between the parentheses on lines 9 and 19, different patterns are matched.</p>
On line 21, the return</code> statement is matched and tells coccinelle to tag
a variable matching i</code>. The <+...</code> and +...></code> snippets tell Coccinelle that
the return</code> statement may not be at a deeper level in the code, like inside
an if</code> block.</p>
Back to those parentheses, the type T</code> can be instantiated with the variable i</code> on line 10, 12.
On line 14 and 16, if the variable i</code> is not using the correct type, then let
Coccinelle do the change and use the type T</code>.</p>
Coccinelle produces this patch:</p>
---</span> main.c
</span></span></span>+++</span> /tmp/cocci-output-1131-72d04b-main.c
</span></span></span>@@</span> -12,7 +12,7</span> @@</span> enum bar {</span>
</span></span></span> enum bar
</span>  my_func(void)
</span>   {
</span>   -    enum foo f = FOO_3;
</span>   +    enum bar f = FOO_3;
</span>
</span>        do_something();
</span></code></pre>
Checks & CI</h1>
Such patches can run in a CI environment and ensure that some APIs are used
correctly.</p>
The following patch ensures that when a function ending with _lock</code>, that is
followed by a function ending with _unlock</code>, then all the code paths have an
_unlock</code> call.  This can find issues that arise typically in error-management
code when one would forget to add an _unlock</code> call.</p>
//</span> A mutex_lock is not matched by a mutex_unlock before an error return/goto.
</span></span>@@
</span>expression l;</span>
</span>identifier LOCK =</span>~</span> "</span>^.*_lock$"</span></span>;</span>
</span>identifier UN =</span>~</span> "</span>^.*_unlock$"</span></span>;</span>
</span>@@
</span>LOCK</span>(</span></span></span>l</span>)</span></span></span>;</span>
</span>...</span> when !=</span> UN</span>(</span></span></span>l</span></span>)</span></span></span>
</span>    when any</span></span>
</span></span>    </span>when strict
</span>(</span>
</span></span>{</span> ...</span> when !=</span> UN</span>(</span></span></span>l</span></span>)</span></span></span>
</span></span></span>+</span>   todo_add_unlock</span>(</span></span></span>l</span></span>)</span></span></span>;</span>
</span></span></span>    return</span> ...</span>;</span>
</span></span></span>}</span></span>
</span></span>|</span>
</span></span>UN</span>(</span></span></span>l</span></span>)</span></span></span>;</span>
</span></span>)</span></span>
</span></code></pre>
It is possible to use regular expression on identifiers to enforce that the
patch only applies to some functions, variables, types.</p>
I wrote a small script to test Terminology’s code against such semantic
patches.  If Coccinelle produces a patch, then the CI fails and one has to
fix the code.</p>
I’ve added some scripts that check the following:</p>

two comparisons of the same expression to different constants</li>
a pointer should not be compared to 0</li>
continue at the end of a for loop has no purpose</li>
calling free() on variables on the stack</li>
!x&y</code>: missing bit-wise operator and negation</li>
NULL</code> tests where the value is known not to be NULL</code></li>
pointer is dereferenced when known to be NULL</code></li>
a variable is only initialized and never used</li>
a variable used an iterator is used outside the iterator loop, without
breaking the loop, and thus the variable has a bad value</li>
missing unlock</li>
missing usage of macros DIV_ROUND_UP</code>, ROUND_UP</code>, MIN</code>, MAX</code>, …</li>
bad type used in return statement</li>
…</li>
</ul>
Sources & Pointers</h1>
Most of the scripts I used can be found on Coccinelle website</a>.
The following links also helped me write some semantic patches:</p>

Official Website</a>.</li>
Presentation: Coccinelle, a bug finding tool</a></li>
Introduction to Coccinelle</a></li>
Introduction to Semantic Patching of C programs with
Coccinelle, by Michele MARTONE</a></li>
Coccinellery</a></li>
LWN - Semantic patching with Coccinelle</a></li>
LWN - Evolutionary development of a semantic patch using Coccinelle</a></li>
Scripts used in the Linux kernel</a></li>
</ul>
Writing Coccinelle patches can be a bit tedious sometimes. Thus, I encourage
anyone who wants to write some semantic patches with Coccinelle, to first
write a test case to validate that the script does indeed what is expected of
it.</p>
I only presented Coccinelle as a tool to find bugs but semantic patches are
also great to automatically refactor some code.</p>


Offline CI
Unknown — Mon, 18 Apr 2022 00:00:00 +0000
In the article about Testing Terminology</a>, I explained
how Terminology is tested and very little about the CI and the concepts I
applied.</p>
The main goal of the CI is to validate the code that will be released at some
point.  By running tests as often as possible, it is possible to catch bugs
soon after they have been written. Using the CI as a gatekeeper, it helps not
introduce them into the final code that will be released. Thus, this should
improve the quality of the releases.</p>
No test is perfect and complete enough to represent the full scale of how the
code will be stressed in the wild and we can not be fully confident that no
bug exist.  However, one major concern is when one can not even rely on the
test results.</p>
Flakiness</h2>
Flakiness occurs when a test is not behaving in a consistent manner.
Usually, it is when a test fails while the tested code has not changed.
Sometimes, it is also when a test passes while it was usually failing and no
change was intentionally introduced to change this behavior.</p>
This produces some important issues:</p>

lack of trust</strong> on the CI: if a test fails, developers tend to dismiss it
and just retry</strong> the test it until it passes.  There are plenty of plugins
for CI frameworks to do just that.</li>
cost in developer time</strong> to investigate the failures.</li>
</ul>
Sources of flakiness</h2>
There are many reasons why a test can fail but let me try to give some common
ones I encountered:</p>

Network issues</em> when interacting with something on the Internet: failing to
fetch a package or when a webservice is down.</li>
Limitations on the test runner</em>: when the test runs on some machine (VMs,
containers, …) that is not behaving as usual. This is usually the case in
shared environments when timings can be very different from one run to
another.</li>
Contract changes</em> with 3rd-parties or the environment: when an API has
changed, or when a package has been modified.</li>
Concurrency</em> issues or timers in the code: this usually reflects a bug in
the code and is one good reason to keep on investigating flakiness issues.</li>
Randomness</em> issues: once in a while, the test hits a bad value.</li>
Badly written tests.</li>
…</li>
</ul>
Learning on retries</h2>
Flakiness is a topic serious enough that Big Tech have teams working on
trying to fix it. Most of the literature is related to handling retries:</p>

This is done at scale at Google</a>.</li>
Azure Pipelines</a>
offers this as a service.</li>
eBay introduced Targeted Auto-Retries</a>.</li>
Facebook wrote a probabilistic model on flakiness</a>.</li>
…</li>
</ul>
All of those are probably great but look far too complex just to test
Terminology.</p>
Flakiness as a risk</h2>
In my personal life, while interacting with the cycling community, I
encountered the concept of the Hierarchy of hazard
controls</a> where,
when wanting to protect cyclists, it is more efficient to separate cyclists
from motor vehicles (risk elimination) than asking them to wear a helmet.</p>

</p>
</div>
I am trying to apply this hierarchy to how flakiness is dealt with in the
industry:</p>

</p>
</div>
To limit the impact of flakiness, the best would be to eliminate</strong> it!</p>
Elimination of some sources of flakiness</h2>
There are different sources of flakiness and some can be removed
systematically.</p>
In Terminology’s CI, fetching dependencies is not a source of flakiness since
all the dependencies are already stored in the image used to compile and run
the tests.  Instead of fetching the dependencies remotely, they are installed
from the image itself.  This is what I call an offline CI</strong>.  It is very
powerful and can enable reproductible builds</strong>.  By constructing the images
outside of the path to test the code, issues found while building those image
do not impact the way the code is tested and clearly decouples</em> testing the
code from creating the environment to test the code.</p>
At some point, Terminology was tested against 4 different versions of the
Enlightenment Foundation Libraries using this system. Terminology is also tested with 2
different compilers (gcc and clang) and many different compilation options
(mostly through of the use of sanitizers).</p>
Terminology’s dependencies are rather small and rely only on alpine packages
but one could imagine using this same concept with cargo, pip, npm, go
modules, …</p>
However, just doing this is not enough.  If a dependency is updated upstream,
this offline CI would not be able to test code against it.</p>
Using the CI to test … the CI</h2>
When the tests are passing and after a review, we tend to consider that the
code being tested is good enough.  These same tests can also be used to
validate a newer dependency: if the code has not changed but only a dependency
has changed, then having the tests green, means that this
newer dependency is working well.</p>
This is represented by the following workflow:</p>

</p>
</div>
When the CI gets complex due to multiple sets of dependencies, it is useful to
try to decouple those sets using image inheritance.</p>
Image inheritance</h2>
Let’s consider the following examples with 3 simple images based on alpine
where fetching (and installing) a different dependency is done on each image:</p>

step1.Dockerfile</code>:  FROM alpine:latest
</span>  RUN apk add --no-cache gcc meson ninja
</span></code></pre>
</li>
step2.Dockerfile</code>: FROM foo:step1-pre
</span> RUN apk add --no-cache netcat-openbsd
</span></code></pre>
</li>
step3.Dockerfile</code>: FROM foo:step2-pre
</span> RUN apk add --no-cache patch
</span></code></pre>
</li>
</ol>
The CI will use the image from step3.Dockerfile</code> when it is validated.</p>
Validating the steps</h3>
There can still be flakiness while building those images, but it’s not
directly impacting the CI used to test changes in the code.</p>
A simple script to build those images and automatically publish when they are
good can be the following one:</p>
#</span></span>!/bin/sh</span>
</span></span>#</span></span> suffix to add to the images. useful to coordinate with change in code</span>
</span></span>SFX</span>=</span>"</span>42"</span></span></span>
</span>#</span></span> branch used to test</span>
</span></span>BRANCH</span>=</span>"</span>master"</span></span></span>
</span>
</span>#</span></span> build Step 1.  Exit early on failure</span>
</span></span>if</span> !</span> LOG1</span>=</span>$</span>(</span>docker</span></span> build --</span>pull</span> --</span>no-cache</span> -</span>q</span> -</span>f</span> step1.Dockerfile -</span>t</span> foo:step1-pre .</span>)</span></span></span></span>;</span> then</span>
</span>    echo</span></span> "</span>$</span>LOG1</span></span>"</span></span></span>
</span>    exit</span></span> 1</span>
</span>fi</span>
</span>#</span></span> build Step 2.  Exit early on failure</span>
</span></span>if</span> !</span> LOG2</span>=</span>$</span>(</span>docker</span></span> build --</span>no-cache</span> -</span>q</span> -</span>f</span> step2.Dockerfile -</span>t</span> foo:step2-pre .</span>)</span></span></span></span>;</span> then</span>
</span>    echo</span></span> "</span>$</span>LOG2</span></span>"</span></span></span>
</span>    exit</span></span> 1</span>
</span>fi</span>
</span>#</span></span> build Step 3.  Exit early on failure</span>
</span></span>if</span> !</span> LOG3</span>=</span>$</span>(</span>docker</span></span> build --</span>no-cache</span> -</span>q</span> -</span>f</span> step3.Dockerfile -</span>t</span> foo:step3-pre .</span>)</span></span></span></span>;</span> then</span>
</span>    echo</span></span> "</span>$</span>LOG3</span></span>"</span></span></span>
</span>    exit</span></span> 1</span>
</span>fi</span>
</span>#</span></span> Run CI on defined branch on image `foo:step3-pre`</span>
</span></span>if</span> !</span> CILOG</span>=</span>$</span>(</span>run_tests.sh</span></span> "</span>$</span>BRANCH</span></span>"</span></span> foo:step3-pre</span>)</span></span></span></span>;</span> then</span>
</span>    echo</span></span> "</span>failed to run tests on newer images: $</span>CILOG</span></span>"</span></span></span>
</span>    exit</span></span> 2</span>
</span>fi</span>
</span>docker</span></span> tag foo:step1-pre "</span>foo:step1-latest-$</span>SFX</span></span>"</span></span></span> &&</span> docker</span></span> push "</span>foo:step1-latest-$</span>SFX</span></span>"</span></span></span>
</span>docker</span></span> tag foo:step2-pre "</span>foo:step2-latest-$</span>SFX</span></span>"</span></span></span> &&</span> docker</span></span> push "</span>foo:step2-latest-$</span>SFX</span></span>"</span></span></span>
</span>docker</span></span> tag foo:step3-pre "</span>foo:step3-latest-$</span>SFX</span></span>"</span></span></span> &&</span> docker</span></span> push "</span>foo:step3-latest-$</span>SFX</span></span>"</span></span></span>
</span></code></pre>
One issue with this approach is that we have to disable the docker cache.  One
improvement could be to only disable it when a dependency is known to have
changed.</p>
By changing the FROM</code> clause on the intermediate images to the one ending
with latest-*</code> and enabling the docker cache, it is possible to test that the
last step is the one failing, and if not, going upward in the steps.  This is
not very practical.</p>
To counter this, one could try to add tests on each step. However, if the
script runs often enough, there should not be a lot of changes in the steps
and finding the root cause of the failure should be easy.  Also, if one step
is clearly broken upstream, using the latest</code> tag for it and enabling the
cache should let test other steps.</p>
When should it run?</h2>
This workflow can run daily or weekly, or be triggered from an external tool
that would alert of an updated dependency, or simply by the need to add or
change a dependency.  In Terminology, this is done every 2 weeks since there
are not a lot of changes in the dependencies.</p>
Sometimes, an incompatibility also needs to be fixed alongside the code.  The
solution is to have, in the code repository, a tag to know which image to use.
In the example, this would be set using the SFX</code> variable. That way, the CI
will only use the newer image when the dependency is known to work well with
the code.</p>
The lifecycle of those images is left as exercice to the reader.</p>
Bonus</h2>
In Terminology, a copy of the repository is stored in the base image and
updated whenever the image has to change.  This can limit the amount of data to
transfer when testing new code.</p>


A dumb raytracer written in Rust
Unknown — Mon, 15 Jul 2019 00:00:00 +0000
Genesis</h2>
A long time ago, I found a computer-generated forest background on
reddit</a>.
I wanted to generate by myself this kind of image and decided few months ago
to write a ray tracer with that goal!</p>
Going back to doing maths in 3d, solving complex systems, triangulate matrices
was hard!!</p>
Rust</h2>
I wrote the raytracer in Rust and making run in parallel was amazingly easy
with rayon-rs</a>.
Since rendering can be quite long, I’ve added a way to stop resume rendering.</p>
Colors</h2>
I wanted to give my rendering an aspect of sunset, during the golden
hour</a> but I couldn’t
find any interesting litterature on it.
I decided that rays from the sun would just mix the objects colors with a bit
of orange and in the end, I like the result.</p>
</p>
The Forest</h2>
I used a Monte Carlo
method</a> to generat my
forest and here’s the result:</p>

4k version</a></p>
While browsing the internets, I went to that
page</a> that
showed the image below (Credits: Photo by Desmond Simon on Unsplash, but
couldn’t find it there).</p>
</p>
My image is no too far from there, with its own style.</p>
As usual, this is opensource and on github</a>.</p>


Testing Terminology
Unknown — Tue, 08 Jan 2019 00:00:00 +0000
Regressions :’(</h2>
A few weeks ago, Egmont Koblinger (a developer on
VTE (a Virtual Terminal library)</a>)
opened a ticket</a> on Terminology about
incorrect parsing of escape codes.
The following day, I pushed a commit to master to fix that issue.</p>
I introduced a regression: some other escape codes were no longer
parsed  correctly. In a few hours after my change, the ticket was reopened and
I got notified twice on IRC.</p>
I felt bad about it, having only tested the specific change; the bug
described in the ticket was fixed and my shell was drawing correctly,
nothing fancier than that.</p>
I quickly reverted my changes on the repository but I needed to act.</p>
Testing tools</h2>
Currently, there is already some testing. On each change to the master
branch, coverity</a> and
codacy</a> do
static analysis. On Terminology itself, I wrote
tyfuzz</a>, a tool to run with
American Fuzzy Lop</a>. I need to run it for a
long period of time (weeks) and it would not help in this case.</p>
In the enlightenment/EFL world, we have
exactness</a>, a pixel
perfect regression tester, but that’s a bit too complex for my needs.
Then, there is vttest</a>, an
VT100/VT220/XTerm test utility. The main issue is that is it interactive: you
press a key then there is a message telling you what you will see and you
need to figure whether what is displayed is right or not. This is would be too
complex to automate.</p>
Introducing tytest</code></h2>
I needed to write something new. I wanted something simple with as little
dependency as possible.</p>
I already had tyfuzz</code> which takes escape codes in input, parses and
interprets them, and just exits.  At first, I wanted to dump some kind of
representation of what is drawn but that would be quite some work.  I ended up
taking a checksum of the different parts of the terminal and output it.
I am using a MD5 checksum</a> as it was
already in use in Terminology.  I write the tests in POSIX shell script.</p>
Here is a simple test that sets a
cursor shape</a>:</p>
    #</span></span>!/bin/sh</span>
</span></span>    printf</span></span> '</span>\033[0 q'</span></span></span>
</span></code></pre>
And how to take a checksum:</p>
tests/cursor-shape-0.sh | build/src/bin/tytest
</span>c91e01b0e859cc043f21d804a01bcd50
</span></code></pre>
I wrote a shell script on top of that simple concept and I am able to test any
new changes for regressions on the tests I already wrote.</p>
Continuous Integration</h2>
I used CircleCI</a> to run the
tests on each change to the master branch of Terminology.
Based on an alpine image</a>,
it compiles Terminology on both clang</code> and gcc</code>, and run the tests.
Then it uploads the coverage of the tests to
codecov.io</a>.</p>
How do I know what to test?</h2>
So far, I have been reading the VT510
documentation</a> and writing
tests based on each command. I write the shell script, run it in XTerm and in
Terminology, then compare visually. Then I fix Terminology ☺</p>
I also rely on the ECMA-48
documentation</a>.
Something I discovered lately is the Terminal
Guide</a>, it is of great help since it
lists bugs in other implementations (urxvt</code>, xterm</code>, vte</code>, konsole</code> and
linux-vc</code>).</p>
I also check the coverage reports from
codecov.io</a>.</p>
Then I try to do some abstract art!</p>
Testing cursor movements:</p>
</p>
Testing IL - Insert Lines:</p>
</p>
At the moment, there are 65 tests which found lots of small bugs.  I will keep
on adding tests and make Terminology even better!</p>


A wrapper to i3status in go
Unknown — Sat, 08 Dec 2018 00:00:00 +0000
I ride my bicycle to go to work and put on my rain clothes? There is a tells you for most places in France 
For every 5 minutes in the next rain and how strong.</p> 
Late last week, I wrote a small public, I just opened Firefox’network query such piece of information. I made the simple script below where file and use flock()</code>  </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span></span></span></span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span></span> </span> </span> </span></span> </span> </span> </span></span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span></code></pre> The idea was to get a string of and put that in my status bar on  
I wanted to write something in On i3status</code>’s output with a custom command:</p> 
</span> </span> </span> </span> </span> </span></code></pre> I did that but I wanted i3bar</code> Here are some screenshots of it rain, then with some:</p> 
</p> 
</p> 
  Terminology supports hyperlinks Unknown — Sun, 14 Oct 2018 00:00:00

one thing is important about it: should I service in France given by MétéoFrance that whether it will rain in the next hour.</p> hour, a rectangle is colored whether it will script in python that uses the API (not page and reverse-engineered it…) to the information is cached in a temporary on it to protect concurrent accesses.</p> z-code">    #</span>!/usr/bin/env python3

import</span> fcntl</span></span></span> import</span> json</span></span></span> import</span> os</span></span></span> import</span> requests</span></span></span> import</span> sys</span></span></span> import</span> time</span></span></span>

TMP_FILE</span></span>=</span>'</span></span></span>/tmp/pluie_dans_lheure.data'</span></span></span>

self</span></span> =</span> sys</span>.</span>argv</span></span></span>[</span></span>0</span></span>]</span></span> fd</span></span> =</span> open</span></span></span></span>(</span>self</span></span>,</span> '</span></span></span>r'</span></span></span>)</span></span> fcntl</span></span>.</span></span>flock</span></span></span></span>(</span>fd</span></span>,</span> fcntl</span>.</span>LOCK_EX</span></span>)</span></span>

class="z-meta z-function z-python">    def</span></span> generate_new_file</span></span></span>(</span></span>)</span></span>:</span></span> BASE_URL</span></span>=</span>'</span></span></span>[REDACTED]'</span></span></span> CODE</span></span>=</span>'</span></span></span>[REDACTED] class="z-source z-python">        r</span></span> =</span> requests</span></span>.</span></span>get</span></span></span></span>(</span>BASE_URL</span></span> +</span> CODE</span></span>)</span></span> assert</span> r</span>.</span>status_code</span></span> is</span> 200</span> j</span></span> =</span> r</span></span>.</span></span>json</span></span></span></span>(</span>)</span></span>[</span></span>'</span></span></span>dataCadran'</span></span></span></span>]</span></span> esc</span></span> =</span> chr</span></span></span></span>(</span>27</span>)</span></span> with</span> open</span></span></span></span>(</span>TMP_FILE</span></span>,</span> '</span></span></span>w'</span></span></span>)</span></span> as</span></span></span> f</span>:</span></span> for</span> p</span> in</span></span> d</span></span></span>:</span></span> n</span></span> =</span> p</span></span></span>[</span></span>'</span></span></span>niveauPluie'</span></span></span></span>]</span></span> if</span> n</span></span> ==</span> 1</span>:</span></span> f</span></span>.</span></span>write</span></span></span></span>(</span>u</span>'</span></span></span>_'</span></span></span>)</span></span> elif</span> n</span></span> ==</span> 2</span>:</span></span> #</span> light f</span></span>.</span></span>write</span></span></span></span>(</span>u</span>'</span></span></span>░'</span></span></span>)</span></span> elif</span> n</span></span> ==</span> 3</span>:</span></span> #</span> medium f</span></span>.</span></span>write</span></span></span></span>(</span>u</span>'</span></span></span>▒'</span></span></span>)</span></span> elif</span> n</span></span> ==</span> 4</span>:</span></span> #</span> dark f</span></span>.</span></span>write</span></span></span></span>(</span>u</span>'</span></span></span>▓'</span></span></span>)</span></span> elif</span> n</span></span> >=</span> 5</span>:</span></span> f</span></span>.</span></span>write</span></span></span></span>(</span>u</span>'</span></span></span>█'</span></span></span>)</span></span>

try</span>:</span></span> st</span></span> =</span> os</span></span>.</span></span>stat</span></span></span></span>(</span>TMP_FILE</span></span>)</span></span> except</span> FileNotFoundError</span>:</span></span> generate_new_file</span></span></span></span>(</span>)</span></span> else</span>:</span></span> t</span></span> =</span> time</span></span>.</span></span>localtime</span></span></span></span>(</span>st</span>.</span>st_ctime</span></span>)</span></span> now</span></span> =</span> time</span></span>.</span></span>localtime</span></span></span></span>(</span>)</span></span> if</span> t</span>.</span>tm_hour</span></span> !=</span> now</span>.</span>tm_hour</span></span> or</span> int</span></span></span></span>(</span>now</span>.</span>tm_min</span></span>/</span>5</span>)</span></span> !=</span> int</span></span></span></span>(</span>t</span>.</span>tm_min</span></span>/</span>5</span>)</span></span>:</span></span> generate_new_file</span></span></span></span>(</span>)</span></span>

with</span> open</span></span></span></span>(</span>TMP_FILE</span></span>,</span> '</span></span></span>r'</span></span></span>)</span></span> as</span></span></span> f</span>:</span></span> print</span></span></span></span>(</span>f</span></span>.</span></span>read</span></span></span></span>(</span>)</span></span>)</span></span> unicode characters representing the next hour href="https://i3wm.org/">i3</a>. the file in /tmp</code>, was taking 1.5s. It was way too much time for such a simple href="https://golang.org/">Go</a> for quite some time to do it.</p> manpage, there is an example about prepending the i3status</code> z-code">    #</span></span>!/bin/sh</span> z-shell z-bash">    #</span></span> shell script to prepend i3status with more stuff</span> z-shell z-bash"> i3status</span></span> |</span> while</span> :</span></span> do</span> read</span></span> line</span> echo</span></span> "</span>mystuff | $</span>line</span></span>"</span></span></span> ||</span> exit</span></span> 1</span> done</span> to display some colors. To do that, it uses a s/i3bar-protocol.html">JSON protocol</a> and thus this small not outputting valid JSON.</p> pushed it on orisfaure/myi3status">myi3status GitHub’s repository</a>. to run but the code is 3 times larger.</p> in action, first when there is no incoming escape sequences
 +0000
Some time ago, Egmont Koblinger wrote a proposal</a> to add support for Hyperlinks in Terminal Emulators.</p>
Terminology already highlights links whenever the mouse hovers them but this
proposal is to act just like links in your web browser.</p>
I’ve patched Terminology to support that. When an application emits the needed
escape codes, Terminology will display some fancy link highlight when you
hover such links. It will also add a tooltip with the actual link so that you
shouldn’t be phished.</p>
Here’s a video of it in action:</p>

	
	
</video>


Terminology provides more choice for cursors!
Unknown — Fri, 16 Mar 2018 00:00:00 +0000
Few months ago, a ticket was opened on
phabricator</a> to add support for
DECSCUSR</code>. This escape code allows to change the cursor style.</p>
I thought it was a good idea since I sometimes feel like the block cursor is
taking too much screen space.</p>
For my 999th commit on Terminology, I’ve added support for this escape code
and did a bit more. It is now possible to choose one of the 3 available
shapes, blinking or not:</p>

Block, the original one,</li>
Underline,</li>
Bar.</li>
</ul>
If you follow the git version, you can try one of the 3 printf</code> below to
change the cursor:</p>
     printf</span></span> "</span>\x1b[1 q\n Block"</span></span></span>
</span>     printf</span></span> "</span>\x1b[3 q\n Underline"</span></span></span>
</span>     printf</span></span> "</span>\x1b[5 q\n Bar"</span></span></span>
</span></code></pre>
If you chose to have a blinking a cursor, it will keep on being blinking. The
same works if you didn’t want it to blink.</p>
Here’s how it renders for me:</p>
</p>


Terminology supports the Encircled escape sequence!
Unknown — Tue, 13 Mar 2018 00:00:00 +0000
TrueColor</h2>
I was looking into TrueColor support for Terminology. It won’t support it
fully but it does approximate those TrueColors to a palette of 256 colors. It
should do it better than XTerm since it is not using a dumb euclidean distance
to compute color distance but something a bit more complex: the last formula
on the Euclidean chapter on Color Distance on
wikipedia</a>.</p>
I looked a bit more on those escape codes
defined in ITU T.416 or ISO 8613</a>
and those TrueColors can be defined as
RGB</code>, CMY</code> and CMYK</code>. Terminology supports all of them (and might be the
only one) but still does color approximation though.</p>
Encircled</h2>
That’s when I stumbled on the Encircled escape sequence.</p>
3 years ago, I added support for the Fraktur escape sequence to
terminology</a>. And I just did the same for Encircled characters.</p>
Unicode provides characters that are encircled in the Enclosed Alphanumerics</a> block.</p>
I did the same
trick</a>:
translate the characters a-z</code>, A-Z</code>, 0-9</code> to their relative enclosed
characters.</p>
If you follow the git version, you can try it like that:</p>
echo "\033[52mabcdefghijklmnopqrstuvwxyz\n\033[0m"
</span>echo "\033[52mABCDEFGHIJKLMNOPQRSTUVWXYZ\n\033[0m"
</span></code></pre>
I’m still using PragmataPro</a>, but it
only supports p</code> enclosed. The other characters are rendered using Symbola
(this is transparent to the user).</p>
Here’s how it renders for me:</p>
</p>
Edit on 2024-01-10</em></p>
I’ve checked with a newer version of PragmataPro</a>, and it renders great:</p>
</p>


The Signin crate
Unknown — Tue, 11 Apr 2017 00:00:00 +0000
For an unreleased personal project, I wanted to support Google SignIn</a>.</p>
With it, you can “Enable Google Sign-In with only a few lines of code”.</p>
On the client side, a web frontend, whenever a user signs in, a token is generated.</p>
Then, this token is passed on all requests to the backend as a way to
authenticate users.</p>
However, I needed to verify the tokens. I couldn’t find anything that suited
my needs so I decided to write a small library for that and publish it as a
crate</a>.</p>
The code is on Github</a>.
I’ve also written a small documentation</a>.</p>


A textual clock on i3lock
Unknown — Tue, 19 Jul 2016 00:00:00 +0000
I decided to give i3</code> a try and it’s a nice ride. One of the things I missed
is my textual clock that I wrote 5 years ago</a>.
I was using it as a lockscreen background.</p>
I looked at the code of i3lock</a> and decided it
was simple to patch it for my needs.
I spent few hours coding and learning about cairo</code> and I’m happy with the result.</p>
This is how it looks:</p>
</p>
It is launched using the following:</p>
i3lock -k -i ~/.bg.png
</span></code></pre>
The font used is Pragmata Pro</a> in
bold.</p>
My patches are available on github</a>.</p>


On mazes with Rust
Unknown — Sat, 06 Feb 2016 00:00:00 +0000
While browsing the internets, I came across a maze as a wallpaper. I wanted to
learn Rust</a> and decided to experiment with Rust by generating mazes.</p>
The result of those experiments is available on
github</a>.</p>
3 Algorithms</h2>
On wikipedia</a>, there
are 3 algorithms.
I started implementing the Prim’s algorithm. It took 295 iterations to
generate this maze:</p>
maze -r plain -g100x100 --foreground #585858 --animation --algorithm prim prim.gif
</span></code></pre>
</p>
Then I worked on Kruskal’s algorithm. Here it needed 287 iterations to
generate this maze</p>
maze -r plain -g100x100 --foreground #585858 --animation --algorithm kruskal kruskal.gif
</span></code></pre>
</p>
Finally I worked on a Backtracker algorithm. This example took 169 iterations.</p>
maze -r plain -g100x100 --foreground #585858 --animation --algorithm backtracker backtracker.gif
</span></code></pre>
</p>
Different rendering modes</h2>
I started with 2-colors mazes:</p>
maze -g630x400 --foreground #ffffff --background #000000 plain.png
</span></code></pre>
</p>
I added different rendering mode based on previous image generation work I
did.
One that look like the background of the blog but not quite good:</p>
maze -g 635x400 -r mosaic mosaic.png
</span></code></pre>
</p>
One about invaders</a>:</p>
maze -g 637x399 --foreground #585858 -r invaders invaders.png
</span></code></pre>
</p>
Bias</h2>
Since the algorithm are based on random number, I wanted to introduce some
bias.</p>
maze -g 630x400 --foreground #585858 -b 0.65 vertical_bias.png
</span></code></pre>
</p>
Here, we can see the impact of the algorithm. It gets more vertical as it goes
to the right starting from the top left corner.</p>
I added an option to specify the starting point of the algorithm.</p>
maze -g 630x400 --foreground #585858 -b 0.65 -o 0.5x0.5 bias_centered.png
</span></code></pre>
</p>
Shading</h2>
These 2-colors images are nice but let’s bring some more colors!</p>
The color goes from red to white based on the length of path from the center:</p>
maze -g630x400 -o 0.5x0.5 --gradient length maze.png
</span></code></pre>
</p>
An other option is to have the solution of the maze drawn and then have a
shading based on how far the cells are from the solution. It generates what I
call the lava river:</p>
maze -g630x400 --foreground #d70000\ #585858 --gradient solution lava_river.png
</span></code></pre>
</p>
Here, we see the impact of the Prim algorithm. It tends to produce mazes that
go from top left to bottom right in a direct way.</p>
With the Kruskal algorithm:</p>
maze -g630x400 --foreground #d70000\ #585858 --gradient solution --algorithm kruskal kruskal.png
</span></code></pre>
</p>
With the backtracker algorithm:</p>
maze -g630x400 --foreground #d70000\ #585858 --gradient solution --algorithm backtracker backtracker.png
</span></code></pre>
</p>
Contributions</h2>
During my development of maze.rs</code>, I wrote 2 crates:</p>

debug_macros</a> to add the macro dbg!()</code> that prints a message (like
println!()</code>) but have it prefixed by the file and line where the call is
written.</li>
color_scaling</a>: function to have
some shading between 2 colors.</li>
</ul>
I also wrote some ebuilds in my gentoo
overlay</a>.</p>
I also found a bug in the rust library I used to generate a gif and patched
it</a>.</p>


Fuzzing Terminology with AFL
Unknown — Wed, 11 Nov 2015 00:00:00 +0000
I wanted to fuzz Terminology for quite a long time. In late August, I found
this
article</a>
about American Fuzzy Lop</a>.  All I had to do
to have it work with Terminology is to have Terminology take its input stdin</code>
(or a file but I chose to read from stdin</code>) instead of a tty</code>.</p>
When epoll</code> is not behaving as I’d like</h2>
I did those changes but afl</code> was not happy because Terminology was crashing
right-away.
I ran my first test-case like that:</p>
    cat</span></span> afl/in/colors_font</span> |</span> src/bin/terminology</span></span>
</span></code></pre>
and it worked like a charm. A window appeared and disappeared with no error
message in the logs. The return code was 0. Nothing misbehaving here, but only
when ran with afl</code>.</p>
Terminology uses epoll</code> to know when there is some new data to
read from stdin</code>. Due to the way afl</code> works, stdin</code> is a file descriptor to a regular
file (set to stdin</code> after fork()</code> before exec()</code> using dup2()</code>). When adding stdin</code> to epoll_ctl()</code>, it replied EBADF</code> saying that stdin</code> is not a valid file descriptor (to epoll</code>) since epoll</code> can not be used to poll regular files.</p>
I did patch afl</code> to make stdin</code> behave like a
pipe</code> for the test application and then it worked (almost) great.</p>
afl</code> did find bugs</h2>
afl</code> did find a bug in Terminology that resulted in this
commit</a>.</p>
It also did find an other bug, but outside Terminology: in Enlightenment! afl</code>
kept on spawning Terminology and thus kept on creating and closing windows. At
some point Enlightenment would crash. This led to
T2803</a> that is now fixed thanks to
zmike</code>.</p>
tyfuzz</code></h2>
Because Terminology had to create windows all the time and initialize a bunch
of stuff, it was slow to actually do the fuzzing. I wrote a small piece of
code called tyfuzz</code> to only test the relevant parts: the parsing of the
escape codes.
I has been running in afl</code> for 2 days but it did not yield any crash so far.</p>
How to run it?</h2>
Everything is currently on git</a>. One need to compile Terminology with the afl compiler companion tool and activate fuzzing.</p>
CC</span>=</span>afl-gcc</span> ./autogen.sh</span></span> --</span>enable-fuzzing</span></span> &&</span> make</span></span>
</span></code></pre>
Terminology produced that way won’t be usable outside afl</code>.</p>
afl-fuzz</span></span> -</span>m</span> none -</span>i</span> afl/in -</span>o</span> afl/out --</span></span> src/bin/terminology -s</span>
</span></code></pre>
or
</span></code></pre>
afl-fuzz</span></span> -</span>m</span> none -</span>i</span> afl/in -</span>o</span> afl/out --</span></span> src/bin/tyfuzz -s</span>
</span></code></pre>


Terminology: on selections
Unknown — Sat, 07 Feb 2015 00:00:00 +0000
I wanted to fix some bugs around selections in Terminology. In order to keep
Terminology rock-solid, I need to test all the ways to make a selection after
my changes to be sure I didn’t break anything. It is time to make a list and
document them.</p>
Usual way to make a selection</h2>
Terminology supports selection by clicking somewhere in the window and
dragging the pointer to where you want the selection to end, while keeping
down the left button of your mouse.</p>
Because Terminology is build with the EFL, a selection is more fancy than just
inversing the color like it is usually the case with other terminal emulators.</p>
Here is what is looks like in action:</p>
</p>
Stretching selections</h2>
One nice feature of Terminology is to be able to adjust selections after they
have been set.</p>
There are 2 anchors that can be moved. They are the red bars on the words
portittor</code> and Aliquam</code>. An animation is played when the mouse hovers the
anchors. Clicking and dragging the anchors allows for expanding or expanding
the selection.</p>
</p>
Introducing the box selection</h2>
Sometimes, the usual way of selecting text is not practical and one would want
to crop the text. The box</code> selection is here in that case.
Press ctrl</code> when doing the selection and the selection will be a rectangle.</p>
</p>
It is possible to stretch/shrink those selections too.</p>
Word and line selection</h2>
Double-click on a word and it will select it.
Triple-click on some text and the whole line will be selected.</p>
</p>
Resizing the selection will now go by word/line.
If you keep the mouse button pressed on your double-click, the selection will
go by words as you drag the mouse, until you release the mouse button.</p>
Other ways to make selections</h2>
Moving the mouse while having the left button of the mouse may not be
practical in some situations. In that case, click somewhere where you would
like to start your selection then move the mouse to where you would like your
selection to end. There, press shift</code> and click and you have your selection.</p>
ctrl-shift+click</code> makes a box selection.
shift+double-click</code> selects text from the original click, up to the word
being double-clicked on. This also works with lines and triple-clicks.</p>
Once there is already a selection, making a shift</code> click will extend the
current selection up to where the mouse pointer is.</p>
From the original selection, this is what happens after a shift-click</code> above
the selection and below it:</p>
</p>
When the application is stealing the mouse?</h2>
Simply do the selection action with shift</code> pressed.</p>
When is all that available?</h2>
It is currently in the master branch of the Terminology repository.
The 0.8</code> version of Terminology will have all those features. It will be
released soon.</p>


Gravatar integration in Terminology
Unknown — Sun, 19 Oct 2014 00:00:00 +0000
A long time ago, I received an email from a friend on my phone through the K9
application. I didn’t have her in my contacts (yet) but it did show me her
picture next to the email. I wonder what was the black magic behind that!</p>
It was just K9 using gravatar</a> to try to fetch a picture
related to her email address.</p>
Since that event, I want to display gravatars as tooltip over email addresses in Terminology.
It can be nice in your email client if you use a console-based one or to see an
avatar after git-blaming someone :)</p>
I finally wrote the code to do that for the EFL dev day 2014.</p>
How does it work?</h2>
It’s simple!
Take an email, let’s say MyEmailAddress@example.com</code>.</p>

Trim the leading and trailing spaces.</li>
Lower-case the address. myemailaddress@example.com</code> in our example.</li>
Take the md5 hash of that string: 0bc83cb571cd1c50ba6f3e8a78ef1346</code> in that
case.</li>
Fetch http://www.gravatar.com/avatar/</code> + the md5sum.</li>
</ol>
How does it look?</h2>
A small capture of live action:</p>
</p>
Of course it is possible to disable that feature in the Behavior panel in the
Settings.</p>
If you have any cool ideas you want to see in Terminology, tell me about them!</p>


Terminology at the EFL Dev Day 2014
Unknown — Tue, 14 Oct 2014 00:00:00 +0000
The 2014 edition of the EFL Dev Day was held in Düsseldorf yesterday.
I gave a quick talk about Terminology.</p>
Just like last year, I used Terminology itself to render my talk.</p>
The code is on github</a>. I refactored it a
bit and it should be more usable than the single script I had last year.</p>
Terminology 0.7</h2>
During the talk I said that the version 0.7 of Terminology would go out soon
and within few hours, it was released.</p>
Distro involvement</h2>
I complained about the state of Terminology in the various Linux distributions.
It is still hard on many distributions to get Terminology.
If Terminology is not available in your distribution, please ask the
distribution maintainers to package it!</p>
The future</h2>
We have some great features planned but I don’t know what features people
would like to see soon in Terminology.</p>
Here is a quick list of possibilities:</p>

Make miniview an awesome scrollbar,</li>
Improvements on CPU usage when on battery,</li>
Searchable buffer,</li>
Zooming on images,</li>
Instant Replay to be able to render the terminal from few seconds ago,</li>
Splits in tabs (Currently tabs can be in splits but not the other way
around),</li>
Have more common “tabs”,</li>
Bold/Italic,</li>
Tmux integration,</li>
…</li>
</ul>
If you have some nice ideas, I would like to hear about them!
Feel free to contact me or open a “Wishlist” task on
phab</a>.</p>
I will probably make a poll about those ideas to know what people think is
important to add to Terminology.</p>
The slides</h2>
Here are the slides:</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>


Miniview of the history in Terminology
Unknown — Thu, 15 May 2014 00:00:00 +0000
Lately I have been working on a new feature for terminology</code>: render the
history in a small widget that I call the miniview. This feature can be found
in trendy code editors.</p>
It started as an old patch from Galatsanos Panagiotis back in June 2013!
I made it work with the current code base and iterated over it.</p>
A picture is worth a thousand words:
</p>
The miniview is shown on the right. It can be shown with hitting Ctrl-Shift-h.
(I plan to make keyboard shortcuts configurable soon, but it will probably not
hit terminology</code> 0.6)</p>
Each character is converted to a single pixel. It is possible to click on it
to scroll the terminal to that position in the history.</p>
The widget can be scrolled up or down with the mouse wheel cause it may not
show the whole history on one screen.  PgUp</code>/PgDown</code> scrolls by steps of 25
lines while Shift-PgUp/PgDown</code> scrolls by a whole screen. Alt-PgUp/PgDown</code>,
Ctrl-PgUp/PgDown</code> do the same.</p>
Hitting Ctrl-Shift-h</code> or the blue cross on the top left corner will close the
miniview.</p>
Now it’s time to fix some bugs to be able to release terminology</code> 0.6 next
month!</p>


Terminology supports the Fraktur escape sequence!
Unknown — Sun, 06 Apr 2014 00:00:00 +0000
Thanks to a tweet from
@resiak</a>, I’ve learnt
that ECMA-48 - Control Functions for Coded Character Sets</a> provides some escape codes to diplay Fraktur!</p>
According to
wikipedia</a>,</p>

Fraktur [fʁakˈtuːɐ] is a calligraphic hand of the Latin alphabet
and any of several blackletter typefaces derived from this hand. The
blackletter lines are broken up – that is, their forms contain many angles
when compared to the smooth curves of the Antiqua (common) typefaces
modeled after antique Roman square capitals and Carolingian minuscule.</p>
</blockquote>
Not long ago, PragmataPro</a>, the
great monospace typeface I use every day got support for such characters.</p>
I’ve spent few minutes to let terminology</code>
handle</a> those escape codes.</p>
If you follow the git version, you can try it like that:</p>
echo "\033[20mabcdefghijklmnopqrstuvwxyz\n\033[0m"
</span>echo "\033[20mABCDEFGHIJKLMNOPQRSTUVWXYZ\n\033[0m"
</span></code></pre>
Mandatory screenshot of the feature:</p>
</p>


gdb + valgrind
Unknown — Sun, 30 Mar 2014 00:00:00 +0000
During Fosdem 2014, I attended a great talk by Philippe Waroquiers about
Combining the power of Valgrind and
GDB</a>. Since the video of
the talk is not available, I wanted to share the power of combining gdb and
valgrind.</p>
Down the rabbit hole</h2>
valgrind</code> can run a gdb server</code> so that a gdb</code> can interact with it.
valgrind</code> has 2 options to trigger that:</p>

--vgdb=yes</code>,</li>
--vgdb-error=0</code>.</li>
</ul>
This second option tells valgrind</code> it needs to wait for that many errors
before freezing and wait for a gdb</code> to connect. Setting it to 0</code> will make
valgrind</code> to wait for gdb</code> to connect before executing the program.</p>
So you start</p>
$ valgrind --vgdb=yes --vgdb-error=0 ./foo
</span></code></pre>
valgrind</code> will tell you to run</p>
$ gdb ./foo
</span></code></pre>
and then enter at the gdb</code> prompt the following:</p>
(gdb) target remote | vgdb
</span></code></pre>
Let’s show some examples.</p>
Leak search</h2>
    #include</span> <</span>stdlib.h></span></span>
</span></span>    #include</span> <</span>string.h></span></span>
</span></span>    
</span>    int</span> main</span></span>(</span></span></span>int</span> argc</span>,</span> char</span> *</span>argv</span>[</span>]</span></span>)</span></span></span> </span>{</span></span></span>
</span></span></span>        char</span> *</span>lost =</span> strdup</span>(</span></span></span>"</span>catch me if you can!"</span></span></span></span>)</span></span></span>;</span>
</span></span></span>        lost =</span> NULL</span>;</span>
</span></span></span>        strdup</span>(</span></span></span>"</span>foobar"</span></span></span></span>)</span></span></span>;</span>
</span></span></span>        return</span> 0</span>;</span>
</span></span></span>    </span></span>}</span></span></span>
</span></code></pre>
Passing commands from gdb</code> to valgrind</code> is done with the monitor</code> keyword.
Showing the help message is as follows:</p>
(gdb) monitor help
</span></code></pre>
Let’s put breakpoints at lines 6, 7 and 8 and continue the execution of the
program.</p>
(gdb) target remote | vgdb
</span>(gdb) breakpoint 6
</span>(gdb) breakpoint 7
</span>(gdb) breakpoint 8
</span>(gdb) continue
</span></code></pre>
Is there a leak?</p>
Breakpoint 1, main (argc=1, argv=0xffeffbe68) at leak_check.c:6
</span>6           lost = NULL;
</span>(gdb) monitor leak_check
</span>==7756== LEAK SUMMARY:
</span>==7756==    definitely lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==    indirectly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==      possibly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==    still reachable: 21 (+21) bytes in 1 (+1) blocks
</span>==7756==         suppressed: 0 (+0) bytes in 0 (+0) blocks
</span>==7756== Reachable blocks (those to which a pointer was found) are not
</span>shown.
</span>==7756== To see them, add 'reachable any' args to leak_check
</span>==7756==
</span></code></pre>
Not yet :) Let’s go to line 7 and lost</code> no longer points to catch me if you can!</code>.</p>
(gdb) continue
</span>Breakpoint 2, main (argc=1, argv=0xffeffbe68) at leak_check.c:7
</span>7           strdup("foobar");
</span>(gdb) monitor leak_check
</span>==7756== LEAK SUMMARY:
</span>==7756==    definitely lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==    indirectly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==      possibly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==    still reachable: 21 (+0) bytes in 1 (+0) blocks
</span>==7756==         suppressed: 0 (+0) bytes in 0 (+0) blocks
</span>==7756== Reachable blocks (those to which a pointer was found) are not
</span>shown.
</span>==7756== To see them, add 'reachable any' args to leak_check
</span>==7756==
</span></code></pre>
Strange! valgrind</code> says the memory is still reachable! Let’s see why!</p>
(gdb) monitor block_list 1
</span>==7756== 21 (+0) bytes in 1 (+0) blocks are still reachable in loss record 1
</span>of 1
</span>==7756==    at 0x4C2C430: malloc (vg_replace_malloc.c:291)
</span>==7756==    by 0x4EB7EF9: strdup (in /lib64/libc-2.17.so)
</span>==7756==    by 0x4005A8: main (leak_search.c:5)
</span>==7756== 0x51E1040[21]
</span></code></pre>
So who is pointing at this part of the memory?</p>
(gdb) monitor who_points_at 0x51E1040 21
</span>==7756== Searching for pointers pointing in 21 bytes from 0x51e1040
</span>==7756== tid 1 register RAX pointing at 0x51e1040
</span></code></pre>
So there is still a register pointing to 0x51e1040</code>.</p>
Go to line 8 and let’s see what changed from line 7:</p>
(gdb) continue
</span>Breakpoint 3, main (argc=1, argv=0xffeffbe68) at leak_check.c:8
</span>8           return 0;
</span>(gdb) monitor leak_check full changed
</span>(gdb) monitor l f c
</span>==7756== 21 (+21) bytes in 1 (+1) blocks are definitely lost in loss
</span>record 3 of 3
</span>==7756==    at 0x4C2C430: malloc (vg_replace_malloc.c:291)
</span>==7756==    by 0x4EB7EF9: strdup (in /lib64/libc-2.17.so)
</span>==7756==    by 0x4005A8: main (leak_search.c:5)
</span>==7756==
</span>==7756== LEAK SUMMARY:
</span>==7756==    definitely lost: 21 (+21) bytes in 1 (+1) blocks
</span>==7756==    indirectly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==      possibly lost: 0 (+0) bytes in 0 (+0) blocks
</span>==7756==    still reachable: 7 (-14) bytes in 1 (+0) blocks
</span>==7756==         suppressed: 0 (+0) bytes in 0 (+0) blocks
</span>==7756== Reachable blocks (those to which a pointer was found) are not
</span>shown.
</span>==7756== To see them, add 'reachable any' args to leak_check
</span>==7756==
</span></code></pre>
The register no longer contains a pointer to 0x51e1040</code>.
There is still a pointer (a register in this case) that points to "foobar"</code>.</p>
Which parts of memory is (un)initialized?</h2>
    #include</span> <</span>stdlib.h></span></span>
</span></span>    #include</span> <</span>stdio.h></span></span>
</span></span>    #include</span> <</span>unistd.h></span></span>
</span></span>    #include</span> <</span>sys/types.h></span></span>
</span></span>    #include</span> <</span>sys/stat.h></span></span>
</span></span>    #include</span> <</span>fcntl.h></span></span>
</span></span>    
</span>    struct</span> s_a</span></span></span> {</span></span></span>
</span></span></span>        char</span> c;</span>
</span></span></span>        int</span>  i;</span>
</span></span></span>    </span></span>}</span></span></span>;</span>
</span>    int</span> main</span></span>(</span></span></span>int</span> argc</span>,</span> char</span> *</span>argv</span>[</span>]</span></span>)</span></span></span> </span>{</span></span></span>
</span></span></span>        struct</span> s_a *</span>a =</span> malloc</span>(</span></span></span>sizeof</span>(</span></span>struct</span> s_a</span>)</span></span></span></span>)</span></span></span>;</span>
</span></span></span>        int</span> fd =</span> open</span>(</span></span></span>"</span>/dev/null"</span></span>,</span> O_APPEND|</span>O_WRONLY</span></span>)</span></span></span>;</span>
</span></span></span>        a-></span>c =</span> '</span>c'</span></span>;</span>
</span></span></span>        a-></span>i =</span> 42</span>;</span>
</span></span></span>        write</span>(</span></span></span>fd,</span> a,</span> sizeof</span>(</span></span>*</span>a</span>)</span></span></span></span>)</span></span></span>;</span>
</span></span></span>        close</span>(</span></span></span>fd</span></span>)</span></span></span>;</span>
</span></span></span>        return</span> 0</span>;</span>
</span></span></span>    </span></span>}</span></span></span>
</span></code></pre>
When running such code in valgrind</code>, we would have the following:</p>
==8592== Syscall param write(buf) points to uninitialised byte(s)
</span>==8592==    at 0x4F103B0: __write_nocancel (in /lib64/libc-2.17.so)
</span>==8592==    by 0x4006CB: main (init.c:18)
</span>==8592==  Address 0x51e1041 is 1 bytes inside a block of size 8 alloc'd
</span>==8592==    at 0x4C2C430: malloc (vg_replace_malloc.c:291)
</span>==8592==    by 0x400688: main (init.c:14)
</span></code></pre>
We can inspect the validity of the memory at get_vbits 0x51e1040</code>:</p>
(gdb) monitor get_vbits 0x51e1040 8
</span>00ffffff 00000000
</span></code></pre>
0</code> means that the bit is valid, 1</code> that it’s invalid.
Here, the padding between c</code> and i</code> is indeed not initialized.</p>
watch</code> on a memory range</h2>
One other great feature of using valgrind</code> with gdb</code> is to be able to have
lots of watchpoints.</p>
Let’s consider this piece of code:</p>
    #include</span> <</span>stdlib.h></span></span>
</span></span>
</span>    int</span> main</span></span>(</span></span></span>int</span> argc</span>,</span> char</span> *</span>argv</span>[</span>]</span></span>)</span></span></span> </span>{</span></span></span>
</span></span></span>        int</span> *</span>array =</span> calloc</span>(</span></span></span>4096</span>,</span> sizeof</span>(</span></span>int</span></span>)</span></span></span></span>)</span></span></span>;</span>
</span></span></span>        array[</span>rand</span>(</span></span></span></span>)</span></span></span> %</span> 4096</span>]</span></span> =</span> 42</span>;</span>
</span></span></span>        return</span> 0</span>;</span>
</span></span></span>    </span></span>}</span></span></span>
</span></code></pre>
Let’s break on line 5. We would like to know when array is modified. We
couldn’t use the watch</code> command in gdb</code> because the number of hardware
breakpoints is very limited (usually 4) and thus it can not cover the size of
the array. valgrind</code> can handle that just fine:</p>
Breakpoint 1, main (argc=1, argv=0xffeffbe68) at watch.c:5
</span>5           array[rand() % 4096] = 42;
</span>(gdb) p array
</span>$1 = (int *) 0x51e1040
</span>(gdb) watch (int[4096]) *0x51e1040
</span>Hardware watchpoint 2: (int[4096]) *0x51e1040
</span>(gdb) continue
</span>Continuing.
</span>Hardware watchpoint 2: (int[4096]) *0x51e1040
</span>
</span>Old value = {0 <repeats 4096 times>}
</span>New value = {0 <repeats 1383 times>, 42, 0 <repeats 2712 times>}
</span>(gdb)
</span></code></pre>
Conclusion</h2>
If you write C code, you must look into valgrind</code> and gdb</code>.
They are really awesome tools for C developers.</p>


View images attachments in mutt within terminology
Unknown — Tue, 03 Dec 2013 00:00:00 +0000
This afternoon some brazilian guys from #develop</code> wanted to render images
attachements in mutt</code> within terminology</code>.</p>
I wanted to do that for quite some time but never took the time to do that.
It is actually quite easy to do that. The only thing needed is the following
line in your mailcap file:</p>
image/*; typop %s && sleep 1; test=test -n "$TERMINOLOGY"
</span></code></pre>
typop</code> only writes some escape codes to let terminology</code> know that it needs to
show a file on popup. The sleep 1</code> is here to make sure terminology</code> has the
time to load the image before mutt</code> removes the file as typop</code> exits as soon
as it has sent the escape codes.</p>
The test</code> part is here to ensure the command is only ran within terminology</code>.
You could then have an other viewer when using mutt</code> from a different terminal
emulator, but who would do that?</p>
This operation only works locally since typop</code> only sends the path to the
file, not the content of the file. This feature could be done in future
versions.</p>
This can also be done for videos since terminology</code> can also render them.</p>
Mandatory screenshot of the feature:</p>
</p>


Terminology at the EFL Dev Day 2013
Unknown — Sat, 26 Oct 2013 00:00:00 +0000
The 2013 edition of the EFL Dev Day was held in Edinburgh last Sunday.
I gave a quick talk about Terminology.</p>
I wanted to make a different talk from the one I gave at
fosdem</a>.</p>
I decided to use Terminology to talk about Terminology and show some of its
awesome features! I wrote a bash script to render my presentation.
With Terminology escape codes, I was able to use real images as bullet points
and inlined animated gifs to entertain my audience.</p>
Escape codes</h2>
This code is now on github</a>. You can run
it yourself in Terminology.</p>
To use images as bullet points, the following code is used:</p>
echo</span></span> -</span>ne</span> '</span>\033}it#2;1;'</span></span>$</span>PWD</span></span>'</span>/item_00.png\0'</span></span></span>
</span>echo</span></span> -</span>ne</span> '</span>\033}ib\0##\033}ie\0'</span></span></span>
</span></code></pre>
Every terminology escape command starts with \033}</code> and ends with \0</code>.
We use it</code> to Insert a Thumbnail on a cell area of W</code> cells wide and H</code>
cells hight. The format is CW;H;FULL-PATH</code> with the following meaning:</p>

C</code> is the replace character to be identified in later text and will be
replaced by the image.</li>
W</code> is the width in character cells (up to 511).</li>
H</code> is the height in character cells (up to 511).</li>
FULL-PATH</code> is the full path of the image to inline.</li>
</ul>
This escape code has no effect by itself. It indicates a future
intention of inserting media into the terminal. Terminology will expect a grid
of WxH</code> “replace characters” to follow. Each sequence of such replace
characters starts with a ib</code> escape command and ends with an ie</code> escape
command.</p>
Other bits</h2>
I used the solarized theme</a>.</p>
The font is PragmataPro</a>.</p>
The slides</h2>
Here are the slides:</p>
</p>
</p>
</p>
![Beware, the list is long04.jpg)</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>


PhotoAlbum.js
Unknown — Sun, 18 Aug 2013 00:00:00 +0000
What is PhotoAlbum.js ?</h2>
It is a small side-project I have started a month ago to show pictures taken
during my holidays.  I could have used one of the many web sites that exist
just for that purpose but I prefer to host my own content.</p>
I had 2 main goals:</p>

generate a static website: I use a very small server and I do not want it
to be overloaded by php/node.js/ruby/… generating web pages.  This is the
same concept I use for this blog.</li>
be able to write a legend about each picture.</li>
</ul>
I decided to go with node.js to build that project.  I know javascript for
quite a long time but never achieved anything with node.js.</p>
Features</h2>

infinite scrolling of the pictures: load more thumbnails as you scroll the
page,</li>
the legend can be written using markdown,</li>
have a timer in the diaporama or change images as you like,</li>
use the html5 history API in the diaporama to make the experience more
natural,</li>
an editor to write those legends and sort or remove images,</li>
…</li>
</ul>
Demo</h2>
I wrote a small demo using pictures from photos-public-domain.com</a>:</p>

the album itself</a>,</li>
the editor</a>.</li>
</ul>
You can find the code on github</a>.</p>
Acknowledgements</h2>
The code uses the following Node.js modules:</p>

imagemagick</a>,</li>
mime</a>.</li>
</ul>
It also uses:</p>

jquery</a>,</li>
markdown.js</a>,</li>
tipsy.js</a>,</li>
jquery-ui</a>, only for the editor.</li>
</ul>
The background can be found at sublepatterns.com</a>.
The icons were modified from the ones at raphaeljs.com</a>.</p>
TODO</h2>
I am currently happy with the state of my photo album generator.
There are still few things that can be improved/added:</p>

use some web-font to improve the look,</li>
play a bit more with exif data: I wanted to put a location mark in
diaporama mode for pictures having some GPS data.</li>
use the EFL instead of imagemagick to speed things up,</li>
have some kind of animation when changing images,</li>
make the diaporama timer more readable,</li>
preload next image in diaporama mode,</li>
have a better support for mobile phones: the diaporama UI should be
changed.</li>
IE support :D</li>
add comments support with disqus and we’ll have a flickr clone :)</li>
…</li>
</ul>


Terminology at FOSDEM 2013
Unknown — Sun, 24 Feb 2013 00:00:00 +0000
I gave a talk at FOSDEM 2013 about Terminology - the great terminal emulator
written with the EFL.</p>
It was not recorded but here are the slides I used:
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
</p>
I used eyelight</code> to make that presentation. You can find the script and the
theme files on github</a>.</p>


Ptyproxy
Unknown — Tue, 22 Jan 2013 00:00:00 +0000
Since few weeks, I have been working on
terminology</a>, a
terminal emulator written with the Enlightenment Foundation Libraries.</p>
I’ve specialized myself on tracking compatibility bugs: since we wrote our own
pty escape codes parser/handler, some bugs slipped into the code and some
applications are not working like on that good old xterm.</p>
Terminology tries to mimic xterm as best as it can, hence $TERM</code> is defined
to xterm</code>.</p>
I was told that emacs (Disclaimer: I’m a proud vim user) wasn’t working
correctly in terminology.
When I launched it, the menu and the status bar were underlined.
I couldn’t figure an issue with the underline code.
Emacs was querying the device id</code> but I thought already fixed that issue some
time ago when I made vim work well in terminology.</p>
On Sunday evening, I decided to write a pty proxy to know what was going on
between emacs and xterm and try to understand why emacs wasn’t giving his love
to terminology. (No, I won’t talk about emacs virgins.)</p>
The code itself is simple: use forkpty()</code> and when the child (emacs in our
case) outputs something through the master</code> file descriptor, write it back to
the parent stdout</code>. When there is something to read from the parent stdin</code>,
write it back to the child stdin</code> through the master file descriptor. I used
poll to do that since it was easy enough for my needs.</p>
On Monday evening the code was working fine and I got 2 files: in.log</code> and
out.log</code>.
Just ran ptyproxy emacs -nw</code> and enter ^X^C</code> in xterm. Backup the 2 files.
Repeat the action in terminology.</p>
The in.log</code> files were different:</p>
xxd terminology.in.log
</span>0000000: 1b5b 3e30 3b32 3731 3b30 6318 03         .[>0;271;0c..
</span>xxd xterm.in.log
</span>0000000: 1b5b 3e34 313b 3238 353b 3063 1803       .[>41;285;0c..
</span></code></pre>
The device id</code> is different. The second number (271 or 285) is xterm’s
version.
The first number denotes the terminal type as expressed below:</p>

0 → VT100</li>
1 → VT220</li>
2 → VT240</li>
18 → VT330</li>
19 → VT340</li>
24 → VT320</li>
41 → VT420</li>
61 → VT510</li>
64 → VT520</li>
65 → VT525</li>
</ul>
Terminology was telling the world it was a VT100 while xterm acts as VT420.</p>
I’ve committed a one-liner to act as xterm in
revision 83047</a> and now emacs
users should enjoy running their favorite text editor in terminology as much
as the vim users already does.</p>
I’ve pushed ptyproxy to github</a>.</p>
If there are still some compatibility issues, please ping me on irc or open a
bug report on our trac</a>.</p>


Why you shouldn't concatenate strings in erlang
Unknown — Tue, 10 Jul 2012 00:00:00 +0000
In the previous article</a>, I said that concatenating
strings was an O(n)</code> operation on the length of the first string.</p>
I feel the need to explain that.</p>
In erlang, strings are lists of integer. Since they are integers, they can
store any unicode characters such as the snowman ☃ which has a value of 9731.</p>
When one wants to concatenate 2 strings, one can use string:concat/2</code>
implemented in
lib/stdlib/src/string.erl</code></a>
as follows:</p>
(All code extracts in this article are subject to the Erlang Public License.)</p>
    %</span>% concat(String1, String2)
</span></span>    %</span>%  Concatenate 2 strings.
</span></span>
</span>    -</span>spec</span> </span>concat</span></span></span>(</span></span></span>String1</span>,</span> String2</span>)</span></span> -></span></span> String3</span> </span>when</span></span></span>
</span></span>          String1</span> ::</span> string</span></span></span>(</span>)</span></span>,</span>
</span></span>          String2</span> ::</span> string</span></span></span>(</span>)</span></span>,</span>
</span></span>          String3</span> ::</span> string</span></span></span>(</span>)</span></span></span>.</span></span>
</span>
</span>    concat</span></span></span>(</span></span></span>S1</span>,</span> S2</span>)</span></span> -></span> S1</span> ++</span> S2</span>.</span></span>
</span></code></pre>
Ok, strings are lists, just concatenate lists!</p>
Down the rabbit hole</h2>
The ++</code> operator is a BIF, a Built-In Function implemented in C in the Beam
VM.
It is implemented in erts/emulator/beam/erl_bif_lists.c</code></a>.</p>
1</td>     /*</span>
</span></span></td></tr>
2</td>      *</span> erlang:'++'/2
</span></span></td></tr>
3</td>      */</span></span>
</span></td></tr>
4</td>     
</span></td></tr>
5</td>     Eterm
</span></td></tr>
6</td>     ebif_plusplus_2</span></span>(</span></span></span>BIF_ALIST_2</span>)</span></span></span>
</span></span></td></tr>
7</td>     </span>{</span></span></span>
</span></span></span></td></tr>
8</td>         return</span> append</span>(</span></span></span>BIF_P,</span> BIF_ARG_1,</span> BIF_ARG_2</span></span>)</span></span></span>;</span>
</span></span></span></td></tr>
9</td>     </span></span>}</span></span></span>
</span></td></tr>
10</td>     
</span></td></tr>
11</td>     static</span> BIF_RETTYPE append</span></span>(</span></span></span>Process*</span> p</span>,</span> Eterm A</span>,</span> Eterm B</span>)</span></span></span>
</span></span></td></tr>
12</td>     </span>{</span></span></span>
</span></span></span></td></tr>
13</td>         Eterm list;</span>
</span></span></span></td></tr>
14</td>         Eterm copy;</span>
</span></span></span></td></tr>
15</td>         Eterm last;</span>
</span></span></span></td></tr>
16</td>         size_t</span> need;</span>
</span></span></span></td></tr>
17</td>         Eterm*</span> hp;</span>
</span></span></span></td></tr>
18</td>         int</span> i;</span>
</span></span></span></td></tr>
19</td>     
</span></span></span></td></tr>
20</td>         if</span> (</span>(</span>i =</span> list_length</span>(</span></span></span>A</span></span>)</span></span></span>)</span></span> <</span> 0</span>)</span></span> {</span>
</span></span></span></span></td></tr>
21</td>             BIF_ERROR</span>(</span></span></span>p,</span> BADARG</span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
22</td>         }</span></span>
</span></span></span></td></tr>
23</td>         if</span> (</span>i ==</span> 0</span>)</span></span> {</span>
</span></span></span></span></td></tr>
24</td>             BIF_RET</span>(</span></span></span>B</span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
25</td>         }</span></span> else</span> if</span> (</span>is_nil</span>(</span></span></span>B</span></span>)</span></span></span>)</span></span> {</span>
</span></span></span></span></td></tr>
26</td>             BIF_RET</span>(</span></span></span>A</span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
27</td>         }</span></span>
</span></span></span></td></tr>
28</td>     
</span></span></span></td></tr>
29</td>         need =</span> 2</span>*</span>i;</span>
</span></span></span></td></tr>
30</td>         hp =</span> HAlloc</span>(</span></span></span>p,</span> need</span></span>)</span></span></span>;</span>
</span></span></span></td></tr>
31</td>         list =</span> A;</span>
</span></span></span></td></tr>
32</td>         copy =</span> last =</span> CONS</span>(</span></span></span>hp,</span> CAR</span>(</span></span></span>list_val</span>(</span></span></span>list</span></span>)</span></span></span></span></span>)</span></span></span>,</span> make_list</span>(</span></span></span>hp+</span>2</span></span></span>)</span></span></span></span></span>)</span></span></span>;</span>
</span></span></span></td></tr>
33</td>         list =</span> CDR</span>(</span></span></span>list_val</span>(</span></span></span>list</span></span>)</span></span></span></span></span>)</span></span></span>;</span>
</span></span></span></td></tr>
34</td>         hp +=</span> 2</span>;</span>
</span></span></span></td></tr>
35</td>         i--</span>;</span>
</span></span></span></td></tr>
36</td>         while</span>(</span>i--</span>)</span></span> {</span>
</span></span></span></span></td></tr>
37</td>             Eterm*</span> listp =</span> list_val</span>(</span></span></span>list</span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
38</td>             last =</span> CONS</span>(</span></span></span>hp,</span> CAR</span>(</span></span></span>listp</span></span>)</span></span></span>,</span> make_list</span>(</span></span></span>hp+</span>2</span></span></span>)</span></span></span></span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
39</td>             list =</span> CDR</span>(</span></span></span>listp</span></span>)</span></span></span>;</span>
</span></span></span></span></td></tr>
40</td>             hp +=</span> 2</span>;</span>
</span></span></span></span></td></tr>
41</td>         }</span></span>
</span></span></span></td></tr>
42</td>         CDR</span>(</span></span></span>list_val</span>(</span></span></span>last</span></span>)</span></span></span></span></span>)</span></span></span> =</span> B;</span>
</span></span></span></td></tr>
43</td>         BIF_RET</span>(</span></span></span>copy</span></span>)</span></span></span>;</span>
</span></span></span></td></tr>
44</td>     </span></span>}</span></span></span>
</span></td></tr></tbody></table></code></pre>
First, the length of A</code> is computed. This operation is O(n)</code> where n</code> is the
length of the list. Trust me on this one, of just go look at
erts/emulator/beam/utils.c</code></a>.</p>
If either lists are empty, return the other one.</p>
On lines 29 - 30, space is allocated on the process heap for a new list. Each
list item has a size of 2 Eterm</code>: one for the list itself, one for the
element.</p>
The code on lines 31 to 42 just copies A</code> to our newly allocated list and
adds B on its tail.</p>
last = CONS(hp, CAR(list_val(list)), make_list(hp+2));</code></h2>
Ok, what is that?</p>
From
erts/emulator/beam/erl_term.h</code></a>,
we need some more definitions:</p>
    #define</span></span> CONS</span></span>(</span>hp</span>,</span> car</span>,</span> cdr</span>)</span></span></span> \</span>
</span></span>            (</span>CAR</span>(</span></span></span>hp</span></span>)</span></span></span>=</span>(</span>car)</span></span>,</span> CDR</span>(</span></span></span>hp</span></span>)</span></span></span>=</span>(</span>cdr)</span></span>,</span> make_list</span>(</span></span></span>hp</span></span>)</span></span></span>)</span></span></span>
</span>    
</span>    #define</span></span> CAR</span></span>(</span>x</span>)</span></span></span>  (</span>(</span>x)</span></span>[</span>0</span>]</span></span>)</span></span></span>
</span>    #define</span></span> CDR</span></span>(</span>x</span>)</span></span></span>  (</span>(</span>x)</span></span>[</span>1</span>]</span></span>)</span></span></span>
</span></code></pre>
make_list</code> is used to return an Eterm</code> tagged as a list from a pointer on
the process heap.</p>
list_val</code> does the opposite and returns the address on the heap of the list.</p>
CONS</code> puts 2 elements in hp[0]</code> (the element in the list) and hp[1]</code> (the next list item) and returns the list at hp</code> as an Eterm</code>.</p>
That complicated expression can now be read as:</p>

copy in hp[0]</code> the element of the list we want to copy,</li>
set the next list item as the Eterm</code> following on the stack (hp+2</code>) since
the new list is allocated as one block on the stack.</li>
</ul>
When list A</code> has been fully copied, the last element is set to B</code> at the
line 42 with CDR(list_val(last)) = B;</code>.</p>
Memory allocation</h2>
If you’re curious, erts/emulator/beam/sys.h</code></a> can tell you
that an Eterm</code> has usually a size of 4 bytes.</p>
Small integers can be stored directly in those 4 bytes.
There are 26 bits usable to store a small integer so it can store up to 2²⁵
= 33554432 because of the sign bit. For reference, pile of
poo</a> which was
added in october 2010 has a value of 128169.</p>
We can consider that unicode characters can be stored directly in the list.</p>
If a string has n</code> characters, it will use 2 * n</code> words.</p>
A binary</code> on the opposite will use 3 to 6 words (depending on the data size)
plus the size of the data itself.</p>
A string stored as a list takes about 8 times the space it would take stored
as a binary</code>.</p>
What about other lists?</h2>
The pointed values are not copied, only the list items themsleves.</p>
Appending a list of records of n</code> tuples to a list of m</code> tuples will take
the same time/memory as concatenating a string of length n</code> to one of length
m</code>.</p>
The only difference is that tuples are not boxed in an Eterm</code>. The list only
contains an Eterm</code> as a pointer</em> to the tuple: where it’s stored in the
heap.</p>
Solution?</h2>
Of course there is one!
You should use binary</code> and IoList</code> whenever possible.
An IoList</code> is iolist = [char() | binary() | iolist()]</code>.
They are “deep-list”.</p>
Want to append an IoList a</code> to an IoList b</code>? Simple! Just create a new list
with 2 elements: a</code> and b</code>!  Concatenation has become an O(1) operation.
Most IO apis accept IoLists.</p>


Tiny Web Framework - a simple web framework written in erlang
Unknown — Mon, 02 Jul 2012 00:00:00 +0000
What is twf ?</h2>
Tiny Web Framework is a very web framework written in Erlang I started working
on out of frustration with Nitrogen.</p>
Why not Nitrogen?</h2>
I enjoy Nitrogen but I don’t use most of its features and I am annoyed by some
of its internals.</p>
For example, I don’t like to have every DOM object created with Nitrogen to be
given a random id or that they automatically have their names added as class
like:</p>
    <</span>table</span> class</span>=</span>"</span></span>table</span> wfid_temp34325</span> wfid_temp34892</span>"</span></span></span>></span></span> … </</span>table</span>></span></span>
</span></code></pre>
I’ve started to patch Nitrogen on
github</a> to fit my needs.</p>
In erlang, strings are lists. That’s handy but it has a cost on memory and
concatenating strings means concatenating lists which is an O(n) operation
where n is the length of the first string, that gets longer each time a string
is appended.</p>
There’s a solution to solve that issue: use lists of binaries aka IoList.
Binaries can be viewed as C-strings.  Want to append an IoList a to an IoList
b? Simple! Just create a new list with 2 elements: a and b!  Most IO apis
accept IoLists.</p>
I was not confortable to change all that in Nitrogen and since I don’t use
a lot of Nitrogen’s features, I decided to created a Tiny Web Framework.</p>
What’s done yet?</h2>
Not much! I only worked on it few hours on my free time.</p>
The current features are:</p>

create HTML using lists of records almost like in Nitrogen</li>
simple templating system</li>
api to get QueryStrings</li>
based on cowboy</a>: well-documented, fast,
handles IoLists…</li>
</ul>
My TODO-list</h2>
Here are the features I need that I should implement for a 0.1 version:</p>

session handling</li>
create more DOM elements</li>
</ul>
There is already a TODO-list for v0.2 which aims at optimization and
documentation:</p>

have a template cache</li>
add spec</li>
add documentation</li>
do not use the process registry to store the context</li>
</ul>


E-Tiling v0.9 is released
Unknown — Sun, 11 Dec 2011 00:00:00 +0000
After many months, I have decided to branch e-tiling v0.9. You can find it on
github</a>.</p>
If you wonder what e-tiling is, I’ve done a presentation in a previous
article</a>.</p>
New Features</h2>
The major change in this release is to be able to tile vertically or
horizontally.</p>
It is now possible to use figures or a different set of characters to identify
windows and transitions.
While talking about settings, the window title is hidden or not as configured.</p>
In order to improve the user experience, the original border and maximized
state are restored when untiling a window.</p>
I’ve also added animations when compositing is disabled.</p>
It has been reported that the module was working with dual-screens
configuration.</p>


Presentation of E-Tiling
Unknown — Mon, 11 Jul 2011 00:00:00 +0000
I’ve branched e-tiling v0.5 on github</a>.</p>
What is it?</h2>
It’s a tiling module for enlightenment.</p>
I’ve done a quick video to introduce it:</p>
</iframe></div>

Making a coat rack inspired by Pablo Picasso

Detect-Changed-Files in Rust with a proper GitHub Actions

Github Action</h2> My ultimate goal was to create a proper GitHub Action around the tool.</p>

Detect-Changed-Files for GitHub Actions

Ronde, a simple monitoring service

Successful proof of concept</h2> I wrote a small proof of concept in Rust that would run some commands and generate a status page based on their output.</p>

Installing Gentoo on a Virtual Private Server

The kernel</h2> I like configuring my kernels and use genkernel</code> to build them. Here it is setup to allow an ssh connection to the initramfs to be able to decrypt the root volume.</p>

Finishing the install</h1> Reboot the VPS, and hopefully enjoy Gentoo.</p>

Note on rewriting the root volume</h2> This may no longer be possible directly in the future starting with kernel 6.8 when the option BLK_DEV_WRITE_MOUNTED</code> will be set to no:</p>

JUnit-Parser v1.0.0

A CLI to control Spotify

Release & Usage</h2> I wrote all this in a tool called spotifyctl</code>. You can find more about it on the following links:</p>

A lovely (env) logger for Rust

Goals</h2> I had two main goals:</p> have the log lines take less space,</li>

State of the library</h2> I have released version 0.6.0 today and I consider it to be fully usable.</p>

Screenshots</h2> Let me show you some examples as screenshots</p> Default output</h3> </p> </div>

Default output</h3> </p> </div>

Output with relative timestamps, short levels (my favorite)</h3> </p> </div>

Output with system timestamps, file names and line numbers</h3> </p> </div>

Useful links</h2> page on crates.io</a></li> documentation</a></li> repository on GitHub</a></li> </ul> I hope it will be useful to some of you.</p>

Coccinelle

Offline CI

Sources of flakiness</h2> There are many reasons why a test can fail but let me try to give some common ones I encountered:</p>

Learning on retries</h2> Flakiness is a topic serious enough that Big Tech have teams working on trying to fix it. Most of the literature is related to handling retries:</p>

Bonus</h2> In Terminology, a copy of the repository is stored in the base image and updated whenever the image has to change. This can limit the amount of data to transfer when testing new code.</p>

A dumb raytracer written in Rust

Testing Terminology

A wrapper to i3status in go

Terminology provides more choice for cursors!

Terminology supports the Encircled escape sequence!

The Signin crate

A textual clock on i3lock

On mazes with Rust

Different rendering modes</h2> I started with 2-colors mazes:</p>

Bias</h2> Since the algorithm are based on random number, I wanted to introduce some bias.</p>

Shading</h2> These 2-colors images are nice but let’s bring some more colors!</p> The color goes from red to white based on the length of path from the center:</p>

Fuzzing Terminology with AFL

Terminology: on selections

When is all that available?</h2> It is currently in the master branch of the Terminology repository. The 0.8</code> version of Terminology will have all those features. It will be released soon.</p>

Gravatar integration in Terminology

How does it look?</h2> A small capture of live action:</p> </p> Of course it is possible to disable that feature in the Behavior panel in the Settings.</p> If you have any cool ideas you want to see in Terminology, tell me about them!</p>

Terminology at the EFL Dev Day 2014

Distro involvement</h2> I complained about the state of Terminology in the various Linux distributions. It is still hard on many distributions to get Terminology. If Terminology is not available in your distribution, please ask the distribution maintainers to package it!</p>

The future</h2> We have some great features planned but I don’t know what features people would like to see soon in Terminology.</p> Here is a quick list of possibilities:</p>

Miniview of the history in Terminology

Terminology supports the Fraktur escape sequence!

gdb + valgrind

Conclusion</h2> If you write C code, you must look into valgrind</code> and gdb</code>. They are really awesome tools for C developers.</p>

View images attachments in mutt within terminology

Terminology at the EFL Dev Day 2013

PhotoAlbum.js

What is PhotoAlbum.js ?</h2> It is a small side-project I have started a month ago to show pictures taken during my holidays. I could have used one of the many web sites that exist just for that purpose but I prefer to host my own content.</p> I had 2 main goals:</p>

Terminology at FOSDEM 2013

Ptyproxy

Why you shouldn't concatenate strings in erlang

Tiny Web Framework - a simple web framework written in erlang

What is twf ?</h2> Tiny Web Framework is a very web framework written in Erlang I started working on out of frustration with Nitrogen.</p>

Why not Nitrogen?</h2> I enjoy Nitrogen but I don’t use most of its features and I am annoyed by some of its internals.</p> For example, I don’t like to have every DOM object created with Nitrogen to be given a random id or that they automatically have their names added as class like:</p>

What’s done yet?</h2> Not much! I only worked on it few hours on my free time.</p> The current features are:</p> create HTML using lists of records almost like in Nitrogen</li> simple templating system</li> api to get QueryStrings</li>

E-Tiling v0.9 is released

Presentation of E-Tiling

Finishing the install</h1>
Reboot the VPS, and hopefully enjoy Gentoo.</p>

Output with system timestamps, file names and line numbers</h3>

</p> </div>
Useful links</h2>

page on crates.io</a></li>
documentation</a></li>
repository on GitHub</a></li> </ul>
I hope it will be useful to some of you.</p>

Useful links</h2>

page on crates.io</a></li>
documentation</a></li>
repository on GitHub</a></li> </ul>
I hope it will be useful to some of you.</p>

`Bonus</h2> In Terminology, a copy of the repository is stored in the base image and updated whenever the image has to change. This can limit the amount of data to transfer when testing new code.</p>`

`When is all that available?</h2> It is currently in the master branch of the Terminology repository. The 0.8</code> version of Terminology will have all those features. It will be released soon.</p>`

How does it look?</h2>
A small capture of live action:</p>
</p>
Of course it is possible to disable that feature in the Behavior panel in the Settings.</p>
If you have any cool ideas you want to see in Terminology, tell me about them!</p>

Distro involvement</h2>
I complained about the state of Terminology in the various Linux distributions. It is still hard on many distributions to get Terminology. If Terminology is not available in your distribution, please ask the distribution maintainers to package it!</p>

Conclusion</h2>
If you write C code, you must look into `valgrind</code> and gdb</code>. They are really awesome tools for C developers.</p>`

What is twf ?</h2>
Tiny Web Framework is a very web framework written in Erlang I started working on out of frustration with Nitrogen.</p>