Due to some constraints, at Ticketsolve we sometimes need to work with an ancient file format: the Paradox Database.

This file format was in use between the 80s and 90s. In order to perform some operations on Paradox databases, nowadays, there are libraries based on the file format reverse engineering work by individual open source programmers, or ad hoc commercial programs.

Additionally, one can use Paradox 7, the reference commercial software originally published in 1992, now abandoned.
This software works good enough in Wine, however, the installer generally raises an error on installation, complaining that there isn’t enough disk space.

In this post we’ll use Radare2, one of the most powerful open source reverse engineering frameworks, in order to statically analyze and patch the installation binary, so that the pesky error is not triggered anymore.

There are no requirements for the reader; knowing x86/32 assembler and the PE executable format will improve the understanding, but it’s not a requirement by any means.

Contents:

• Disclaimer
• Tools
• Problem
• Finding a starting point
• Examining the relevant function
• Deciding and approach, testing the theory, and patching the binary
• Digging further
• Taking the simplest approach, and creating the final patch
• Distributing the patch
• Conclusion

I wanted to tweak a script of mine; it included the conventional find </path> -maxdepth 1 -type f. I wasn’t fully convinced it was the best choice, so I checked out what’s the ls equivalent.

This post is about the research I’ve done; as usual, it’s an exercise in extensive usage of the available tools.

Contents:

• Introduction to the problem, and the find tool
• Exploring ls
• Conclusion

We’re in the process of upgrading our MySQL databases from v5.7 to v8.0; since one of the differences in v8.0 is that the default encoding changed from utf8 to utf8mb4, and we had the conversion in plan anyway, we anticipated it and performed it as preliminary step for the upgrade.

This post describes in depth the overall experience, including tooling and pitfalls, and related subjects.

Contents:

• Introduction
• Migration plan: overview and considerations
  • Free step: connection configurations
    • How do charset settings affect database operations?
  • Step 2: Preparing the the ALTER statements
    • Issue: Column/index size limits
    • Issue: Triggers/Functions
    • Issue: Optimization problems with joins between columns with heterogeneous charsets
  • Step 3: Altering the schema and tables
  • Warnings
    • Other schemas
    • Always run ANALYZE TABLE
    • Don’t rush the DROP TABLE
• Notes about Mathias Bynens’ post on the same subject
• Conclusion
• Footnotes

We recently had to drop a column in production, from a relatively large (order of 10⁷ records) table.

On modern MySQL setups, dropping a column doesn’t lock the table (it does, actually, but for a relatively short time), however, we wanted to improve a very typical Rails migration scenario in a few ways:

1. offloading the column dropping time from the deploy;
2. ensuring that in the time between the column is dropped and the app servers restarted, the app doesn’t raise errors due to the expectation that the column is present;
3. not overloading the database with I/O.

I’ll give the Gh-ost tool a brief introduction, and show how to fulfill the above requirements in a simple way, by using this tool and an ActiveRecord flag.

This workflow can be applied to almost any table alteration scenario.

Contents:

• Gh-ost
• Setup and workflow
  • Existing configuration
  • Configure ActiveRecord for ignoring the column, and performing the deploy
  • Using gh-ost to drop the column
  • Remove the ignored_columns and redeploy
• Conclusion

The bindings for the standard Ruby GUI toolkit, Tk, need some trickery in order to be installed on Ubuntu. This article shows how to do it.

Contents:

• Some history
• Prerequisites
• Workaround and installation

I’ve recently moved from Bash to Zsh, and I needed to port my tab completion scripts. Zsh has a sophisticated built-in tab completion, however, the documentation is not very beginner-friendly; moreover, Bash scripts can be used with no or little change in Zsh. Therefore, I’ve opted for using them directly.

This article will explain how to write tab-completion scripts in any language, with an example in Ruby, and how to use them in both Bash and Zsh.

As typical of this blog, the script is also used as an exercise in shell scripting, therefore, it contains additional (arguably) useful/interesting commands/concepts.

Contents:

• General notes
• Ruby Library
• Writing a tab-completion script
• Specification of the target program
• Writing the tab completion script
• Setting up autocompletion
• Debugging
• Conclusion

Seldom, the subject of how to wipe the disk(s) of a headliness linux server comes up; there are a few resources online about it. This blog summarizes all the information around into a clean, stable and generic script that can be used in order to perform this task.

As typical of this blog, the script is also used as an exercise in shell scripting and system administration, therefore, it contains (arguably) useful/interesting commands/concepts.

Contents:

• Preamble/specifications
• Procedure

Although the concepts involved in preparing a Debian (.deb) source package and publishing it on an Ubuntu PPA are simple, due to the many moving parts involved, it’s not easy to find a single source of information.

This article provides all the information required to perform the process, using a trivial program as an example.

A second article will introduce other concepts, using a Golang program as an example.

Contents:

• About the approach and standards
• Conventions
• The procedure
  • Setup
    • Preparing the system
    • Setting up and publishing the PGP key(s)
    • Setting up a Launchpad account
    • Creating the PPA
  • Preparing the source package
    • Introduction to Debian packaging
    • Preparing the source code
    • The makefile
    • Debian packaging metadata creation and core concepts
    • Updating changelog
    • Updating control
    • Updating copyright
  • Building the source package
  • Uploading the package
  • Deleting a package
• Using the PPA
• Conclusion

With the large diffusion of SBCs [Single Board Computers], and subsequent maturation of their ecosystem, it’s now relatively easy to setup a home server.

I’ve had three SBCs until now; a Raspberry Pi 2 model B, a 3 model B, and recently, an Odroid XU4.

In this post, I’m going to share some considerations about their usage as home servers.

Updated on 25/Oct/2018: added new RK3399 boards, and updated prices.

Contents:

• General characteristics of an SBC/home server
• Brief informations about ARM processors
• Raspberry Pi 3 Model B
  • Specifications
  • Support and documentation
    • BEWARE: Stay far from the Ubuntu Pi Flavour Maker
  • Usage impressions
• Odroid XU4
  • Specifications
  • Support and documentation
  • Usage impressions
  • Power draw
  • The infamous fan noise
    • Introduction
    • Base setup
    • Solutions and references
  • Performance tweaking
• Alternatives
• Benchmark
• Conclusions
• Footnotes

In system administration, it’s typical to perform long-running commands on remote hosts.

With GNU Screen and one supporting script, it’s possible to efficiently perform such operations, by running them asynchronously and receiving an email at the end, all in a single command.

Contents:

• Some GNU Screen concepts
• Purpose and overview of the script
• Script requirements
• Script source
• Script explanation
• Conclusion
• References

Ruby GUI development is a seldom mentioned subject, but it has value. Probably after some Rails development (cough…), developing a desktop tool may be an interesting diversion (or even a requirement).

During the development of my PM-Spotlight desktop application, I evaluated most of the Desktop Ruby GUI toolkits, and prototyped the application with three of them (Shoes 3, FXRuby, and Tk).

This article presents a summary of what I’ve experienced (or gathered) while I was “Developing GUI applications with Ruby”!

Contents:

• TL;DR: The summary
• Extension/mistakes
• Frameworks
  • Shoes 3/4
  • FXRuby
  • Ruby-GNOME2
  • qtbindings
  • Tk
  • wxRuby
• Conclusion
• Some references
• Footnotes

It’s very convenient to run service processes (for development purposes!) without admin permissions, rather starting it as system service.

This guide will show how to easily setup PostgreSQL (both via package and binary tarball) to be run by an unprivileged user, with the data in any directory owned by him/her.

Contents:

• Setup notes/requirements
• Installation via apt package (Ubuntu)
• Installation via binary tarball (universal)
• Persisting the PGSQL data location, and adding the binaries to the $PATH
• Conclusion

GitLab offers a functionality for cherry picking a merge request (PR).

This functionality doesn’t exist in GitHub, and in Git, either; it is useful in some cases.

In this article I’ll explain some git fundamentals, and in the last section, how to cherry pick a PR/merge request.

Contents:

• Disclaimer
• What is a git merge
• Rebasing onto a new base
• Finding the merge base
• Cherry-picking a PR/merge request
• Merging the PR
  • Maintaining the PR exactly as it is
  • Changing the PR branch history
• Conclusion

In a project of mine, I’m implementing a feature that runs a background job in order to perform a search; in particular, it needs to support stopping at any time.

There is a variety of strategies to do this in Ruby. In this article I will expose what is the exact outcome of the common strategies, examining how this affects the underlying operating system.

The analysis is targeted to POSIX operating systems, although, at least part of it applies to Windows machines as well.

Contents:

• Brief introduction to Ruby concurrency
• Technical context and preliminary notes
• Problem statement
• Effect of the common strategies on a Linux operating systems
  • Threads, and Thread#kill
  • Kernel#fork
  • Dealing with zombie processes: Process.detach
  • Kernel#spawn
• Introduction to process groups and their usage
  • Non-working usage of process groups
  • Working implementation, using setsid
• Conclusion

This is the Part 3 (of 3) of the shell scripting adventures.

The following subjects are described in this part:

• Introduction to Whiptail
• Widgets, with snippets
  • Message box
  • Yes/no box
  • Gauge
  • Radio list
• Other widgets
  • Input box
  • Text box
  • Password box
  • Menus
  • Check list

Since Whiptail is simple to use, the objective of this post is rather to show some useful code snippets/patterns.

The examples are taken from my RPi VPN Router project installation script.

Previous chapters:

• Introduction
• Part 1
• Part 2

This is the Part 2 (of 3) of the shell scripting adventures.

The following subjects are described in this part:

• Awk/sed/perl considerations
• Perl text processing
  • Regex modifiers
  • Replace multiple lines
    • m modifier
    • s modifier
  • Isolating ambiguous group references
  • Printing a matching group (of matching lines)
• Awk text processing
• Progress bars processing with awk (and stdbuf)
  • Processing single-line progress outputs
  • Working around the buffering
  • Streams processing
  • Putting things together

The examples are taken from my RPi VPN Router project installation script.

Previous/following chapters:

• Introduction
• Part 1
• Part 3

This is the Part 1 (of 3) of the shell scripting adventures.

The following subjects are described in this part:

• Associative arrays (hash maps)
• ANSI-C quoting
• Escape strings
• Expand strings into separate options
• Regular expressions matching
• Find a filename’s basename
• Replace the extension of a filename
• Cycle a multi-line variable
• Heredoc

The examples are taken from my RPi VPN Router project installation script.

Previous/Following chapters:

• Introduction
• Part 2
• Part 3

I’ve always thought of shell scripting as a second class scripting form, being awkward and limited. After a deep dive in my RPi VPN router project, I still think it’s awkward and limited, but I do appreciate it as first class choice under specific conditions.

Shell scripting generally can’t be disassociated from system/infrastructure administration, so developing the project has been actually, as a whole, a very interesting and pleasant undertaking.

This post lays the structure of the future posts I’ll publish about the experience.

Following chapters:

• Part 1
• Part 2
• Part 3

Mysql 5.7 added native support for JSON data type. This opens up several interesting possibilities, but it’s not natively supported in Rails 4 (only in v5).

I’ve released a gem, JSON on Rails, for supporting this functionality on Rails 4.

This article describes how the gem works, for those interested in the inner workings, or who want to implement the functionality by themselves.

MySQL 5.7.19 fixes a quite dangerous functionality that causes corruption of a slave when changing the delay (CHANGE MASTER TO MASTER_DELAY=<seconds>) while the slave threads are stopped; since one wouldn’t expect this condition to cause harm, users of such setup should upgrade, if possible.

Users trying to perform the update on Ubuntu 14.04 LTS will face the mysql service not starting.

In my VGA Passthrough guide, I explain how to configure an Ubuntu machine for VFIO.

With the release of QEMU 2.10, some bugs in the OVMF firmware surfaced, which make Windows guests unstable; such bugs have been fixed in the master branch of the EDK II Project repository, so, in order to use QEMU 2.10, it’s required to build the firmware from scratch.

In the vein of the previous article, we’ll examine here the impact of using the ENUM data type.

Such data type is very interesting, although it requires a careful examination, since it has very important design implications.

Recently, we decided to clean our data model, in particular, the OLTP section of our schema. We have a reasonably well structured model, but it falls short when it comes to the data type definitions.

In this article we explore the storage requirements of a (InnoDB) table of ours, and the impact of choosing more strictly defined columns.