Some history

In a previous post I studied the efficient conversion from little to big endian and back, this post will demonstrate a CPU time efficient method to convert floating number from one format to another. Before the definition of IEEE754 various representations for floating numbers existed. Typically a different approach to the presence, size and base of the exponent existed. Also the size of the fraction part and how to deal with the redundancy of having representations for both +0 and -0, as also various methods to detect over- and underflow or positive and negative infinity. To perform calculations with these different kind of formats would be very time consuming if a microprocessor has no hardware support for it.  Due to the existence of IEEE754 however,  nowadays almost every floating number is represented in this format, backup by efficient arithmetic hardware in your PC.  However one type of representation deviating from IEEE754 for floating number this exist today: GDSII.

Calma’s GDSII format

A still popular file format to store graphical layout of microelectronic circuits is the Graphic Database System (GDSII), originally defined by the company Calma. A thorough description of the format is given here. I will only focus on the used representation of the floating number in this format. The most significant bit (MSB) is the sign bit, where ’1′ is chosen as representation of the minus sign. After this bit, the next seven bits represent the exponent of the number, however with an offset of 64. The remaining 56 bits are the fraction. Using the character s, e and f for respectively sign, exponent and fraction, the bit positions have the following functions:

Calma's GDS format:
seeeeeee ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff

IEEE 754 format:
seeeeeee eeeeffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff

Compared to IEEE754 definition, not only the sizes of exponent and fraction differ, also the base of 16 makes direct comparison difficult. To further complicate things: the IEEE754 has used a handy trick to use the 64-bit word very efficiently: the leading ’1′ of the fraction is not included for normal numbers (only for the very small subnormals). The total value is now:

For comparison, the IEEE754 value is:

From this we can conclude that the range of presentable numbers is much larger in the IEEE754 format having an exponent varying from -1023 till 1024, but Calma’s GDSII format is a four bits more accurate having 56 instead of 52 to store the fraction part.

Conversion from Calma’s GDSII to IEEE754

Not surprisingly both formats have a power of 2 as base, which means that all multiplications by powers of 2 can be done by shifting bits left or right. Now assume b is a pointer pointing at an array of eight characters. To convert this to a 64-bit register value containing the last 56 bits an bitwise AND is performed like this:

uint64_t fraction = be64toh(*((uint64_t*) b)) & 0xFFFFFFFFFFFFFF;

The function big endian to host, be64toh(), is introduced by the following pre-processor statement: #include <endian.h>. This ensures that regardless the endianness of the host computer, the sequence of 8 bytes in memory, to which b is pointing, will be interpreted as being in big endian format.

signed char exp_gdsii = ((*b) & 0x7F) - (64);
uint16_t exp_ieee = 1023+(((uint16_t) exp_gdsii) << 2);

The exponent of GDSII format is extracted from the first of the 8 character bytes and bitwise OR with one zero and seven ones to remove the possible sign bit. Then it is shifted twice, to counteract the representation change from base 16 to base 2. Finally the offset of 1023 is added, see the code above. After doing this, the leading ’1′ has to be found, and will be suppressed in the IEEE754 format:

int count = 0;
while (((fraction & 0x100000000000000) == 0) && count <= 56) {
fraction <<= 1;
exp_ieee-=1;
count++;
}
if (count >= 57) exp_ieee = 0;
fraction &= 0xFFFFFFFFFFFFFF;
fraction >>= 4;

The while loop above will search for the first ’1′ to occur, but to prevent and endless loop, never more than 56 times. Each iteration of the loop the fraction bits are right shifted causing a multiplication by 2, which must be undone by lowering the exponent one step. Now the leading ’1′ is present at the 56th bit position, when starting with 1 as LSB, an can effectively be suppressed by an bitwise AND function of the lower bits. And then the 56 fraction will be right-shifted four times, as the IEEE754 format has only place for 52 bits. Next, everything (sign, exponent and fraction) must be combined:

fraction |= (( (uint64_t) exp_ieee) << 52);
if((*b) & 0x80) fraction |= 0x8000000000000000;
return *((double*) (&fraction));

Therefore the fraction variable is binary OR-ed with a 52 times left-shifted copy of the exponent. And if the original number contained a minus sign, the new IEEE754 also should. The fraction is 64-bit wide unsigned integer type, but has to be interpreted as a double by the compiler: to achieve this the address of the fraction is interpreted as a pointer to a double, and the return statement return the double type this pointer is pointing too.

Conversion from IEEE754 to Calma’s GDSII

The other way around goes in a similar way. However, as an IEEE754 number can contain number much larger than 63th power of 16 , all powers of 2 above 252 can only be clipped to the largest value in GDSII format. Not a real problem in daily use, as your layout most likely doesn’t describe an area of several galaxies. At the other end of the exponent range, having a power of -256 in base 2, will be rounded to a real zero in GDSII. As long as sub-micron electronics is limited by the spacing of atoms, this is neither a problem, if correctly set to a real zero. Therefore, before doing a real conversion, the extreme large and small number are filtered out by this piece of code:

if (exp_ieee > 1275) {
fraction  = uint64_t(0x7FFFFFFFFFFFFFFF);
exp_gdsii = 0x7F;
} else if (exp_ieee < 767) {
fraction  = uint64_t(0x0);
exp_gdsii = 0x40;
} else {
...
}

Between the curly brackets of the last else statement the real conversion procedure can be handled. The exponent of the IEEE format is copied, its initial offset of 1023 is subtracted and a new offset of 64 times 4 is added. Why? Because 16 is the 4th power of 2. And to handle the difference in amount of bits of the fraction part, 52 instead of 56, the new exponent is incremented by 4.

As factors 2^1, 2^2 and 2^3 do not fit in 2^4, shift fraction
*  accordingly, if needed. *//** Looped twice, thus effectively exp is divided by four */


exp_gdsii = exp_ieee - 1023 + 4 + 4*64;
for(int i=1; i<3;i++)
{
if(exp_gdsii & 0x1) {
fraction <<= i;
}
exp_gdsii >>= 1;
}

This new value of exp_gdsii stil needs to divided by four. This can be done easily by rightshifting its bits twice, but we need to take care of the bits which will fall of, if present. Therefore shifting is done in two iteration of a for loop, each time shifting the exponent bits one position. And, if a LSB was detected also shift the fraction to the left. In the first iteration of the loop this LSB will represent a single power of 2: so we shift the fraction once. In the second iteration of the for loop, the LSB of the exponent represents a double power of 2: now we must shift the fraction twice. This small but distinctive difference is handled by using the loop variable i as argument of the shift operation: first shift once, next time shift twice, if required.

Full range functional test

To test the procedure a test is written which offers IEEE754 numbers starting a little bit above the maximum value in Calma’s GDSII till beyond the smallest possible non-zero value. This largest number is divider every time by a negative number, so automatically the sign functionality is tested too. Some output:

ieee[le] 11111111 01110100 01100010 11011111 01111111 00000001 01111111 00111110     1.1550581988309e-07
gds [be] 00111011 00011111 00000001 01111111 11011111 01100010 01110101 00000000 K   1.1550581988309e-07
gds [be] 00111011 00011111 00000001 01111111 11011111 01100010 01110100 11111111 ?   1.1550581988309e-07
ieee[le] 11111111 01110100 01100010 11011111 01111111 00000001 01111111 00111110 K   1.1550581988309e-07
ieee[le] 11111111 01110100 01100010 11011111 01111111 00000001 01111111 00111110 ?   1.1550581988309e-07

The first row contains a sequence of bytes, which represent a little endian IEEE754 value of 1.1550581988309e-07. This number is converted, on the second row, using the write_double() function in klayout’s dbGDS2Writes.cc source code file an act as a functional verification of the quicker shifting procedure ieee754_to_calma(), present on the third line. There is a tiny difference in the last nine bits: a rounding difference due to the many complex operation is the write_double() function. Its shifting equivalent hasn’t surely touched them! So its quicker, and even a little bit more accurate, in this case at least. The fourth and fifth row represent the output of the conversion from GDSII to IEEE754 for respectively procedures get_double() and calma_to_ieee754(). No difference, just gaining speed.

ieee[le] 00000000 00000000 00000000 00000000 00000000 00000000 10110000 01001111     7.2370055773323e+75
gds [be] 00000000 00010000 00000000 00000000 00000000 00000000 00000000 00000000 K   5.3976053469340e-79
gds [be] 10000000 00010000 00000000 00000000 00000000 00000000 00000000 00000000 ?  -5.3976053469340e-79
ieee[le] 00000000 00000000 00000000 00000000 00000000 00000000 10110000 10101111 K   7.2370055773323e+75
ieee[le] 00000000 00000000 00000000 00000000 00000000 00000000 10110000 10101111 ?   7.2370055773323e+75

At the top side of the fraction range both conversion methods fail, although in a different way. But as long as you do not route galaxy sized metal tracks on your chip design it works already for the next smaller number (and again a tiny rounding difference):

ieee[le] 00101001 00000110 10100001 10011001 00001011 00100000 01110010 11001111    -5.1239065260070e+74
gds [be] 11111111 00010010 00100000 00001011 10011001 10100001 00000110 00101010 K  -5.1239065260070e+74
gds [be] 11111111 00010010 00100000 00001011 10011001 10100001 00000110 00101001 ?  -5.1239065260070e+74
ieee[le] 00101001 00000110 10100001 10011001 00001011 00100000 01110010 11001111 K  -5.1239065260070e+74
ieee[le] 00101001 00000110 10100001 10011001 00001011 00100000 01110010 11001111 ?  -5.1239065260070e+74

On the lower side of the fraction range, accuracy stops far below the distance of atoms:

ieee[le] 11000000 00100010 11001101 00000001 11110100 11100111 00101010 00110000     1.1618266489153e-76
gds [be] 00000001 11010111 00111111 10100000 00001110 01101001 00010110 00000000 K   1.1618266489153e-76
gds [be] 00000001 11010111 00111111 10100000 00001110 01101001 00010110 00000000 ?   1.1618266489153e-76
ieee[le] 11000000 00100010 11001101 00000001 11110100 11100111 00101010 00110000 K   1.1618266489153e-76
ieee[le] 11000000 00100010 11001101 00000001 11110100 11100111 00101010 00110000 ?   1.1618266489153e-76

One step further is really zero:

ieee[le] 01010000 11111000 11011010 11101011 11010101 01111010 11101110 10101111    -8.2259037731186e-78
gds [be] 11000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 K  -0.0000000000000e+00
gds [be] 11000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 ?  -0.0000000000000e+00
ieee[le] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 10000000 K  -8.2259037731186e-78
ieee[le] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 10000000 ?  -8.2259037731186e-78

Practical speed improvement test

Assume we may create chips with dimension up to 20 millimeter using an accuracy of 5 nanometer, a set of 117 numbers is defined in this range, spaced logarithmically with equal distance. Also the sign is switched in every next number, and the ratio is neither a positive or negative power of 2. So the bits vary quite randomly, as in a real application. In code:

struct p {unsigned char word[9]; } calma_pattern[117];
double ieee754_pattern[117];
double x = 20e-3;
double r = -pow(x / 5e-9, 1/(-1+117);
for(int i = 0; i < 117; i++)
{
ieee754_pattern[i] = x;
write_double(x,calma_pattern[i].word);
x /= r;
}

These number will be converted 1 million times to get a reasonable accurate test, suppressing effects of cache latency and rounding errors in time samples. The original conversion functions of klayout (release 0.21.14) using mathematical pow() and log() functions requires 46.08 seconds in an example run to fulfill its duty. Using the new proposed shift operations only takes 1.41 seconds: shifting is 32.6809 times faster, saving 44.67 seconds in this example run!

To convince yourselves, you can download the code here (including the modified klayout source files).

Once you start writing applications for all type of processor architectures, you’ll bump into the problem of endianness. Biased with the popular style of Arabic numbers, you most likely assume that the most significant byte, of a four byte integer, will be printed first: big endian, as the biggest byte is printed first. This is indeed the case for some CPU’s however, the very popular x86 and thereof derived amd64 processors use little endian instead: the byte with least value first, followed by the other with incremental value. A possible reason, I guess,  for using little endian could have been saving clock cycles during an addition operation when reading from memory: as the lowest byte arrive first, processing can start immediately. With nowadays processors capable of doing even a integer multiplication in 3 clock cycles (see the manual), this not an issue anymore.

In a high level programming language like C/C++ you would normally not be bothered with the actual implementation of endianness, however, when sharing binary data (e.g. files, network traffic) between different systems the endianness issue pops up. Software originally written for big endian architectures automatically define their format using this type of endianness. Reading and writing such data in little endian machines therefore require a conversion before further interpreting the content of the file.

Looking around on the web, luckily the problem has been solved earlier, see e.g. here or here. It  requires writing some assembly code inside your C/C++ function, or even simply calling a built-in function of the GNU C compiler.

Built-in byte swap for 4 and 8-bytes words

To demonstrate what happens when calling the built-in functions, a character array of nine bytes, originally containing “abcdefgh” and a ‘\0′ end-of-string character, is created and then overwritten with a 4-byte assignment of an 32-bit integer. The integer is a sum of characters multiplied with a third, second, first and zeroth power of the byte value 256:

int *ptr32b, num32b = 16777216*'U'+65536*'N'+256*'I'+'X';
*ptr32b = num32b;
printf("Default 4-byte sequence (little endian) : %s\n",array);
*ptr32b = __builtin_bswap32 (num32b);
printf("Swapped 4-byte sequence (big endian) : %s\n",array);

The first printf() statement will print “XINUefgh” instead of what you might expect when biased by the notation of decimal numbers.  The result “UNIXefgh” is achieved by calling the built-in swap function, as processed by the second printf() statement. To verify if really a special machine code is used, first compile the byte swap example file, and then disassemble it using the following shell commands:

gcc -g byte_swap_demo.c -o byte_swap_demo
objdump -Dslx byte_swap_demo > byte_swap_demo.asm

The content of the disassembler’s output reveals indeed the usage of the byte swap instruction (which was available since the i486 processor):

40066f:    48 8b 45 e0              mov    -0x20(%rbp),%rax          (line 64)
400673:    8b 55 f8                 mov    -0x8(%rbp),%edx
400676:    89 10                    mov    %edx,(%rax)
400678:    b8 90 08 40 00           mov    $0x400890,%eax            (line 65)
40067d:    48 8b 55 f0              mov    -0x10(%rbp),%rdx
400681:    48 89 d6                 mov    %rdx,%rsi
400684:    48 89 c7                 mov    %rax,%rdi
400687:    b8 00 00 00 00           mov    $0x0,%eax
40068c:    e8 bf fd ff ff           callq  400450 <printf@plt>
400691:    8b 45 f8                 mov    -0x8(%rbp),%eax           (line 68)
400694:    0f c8                    bswap  %eax                  <-- Ah, there it is!!
400696:    89 c2                    mov    %eax,%edx
400698:    48 8b 45 e0              mov    -0x20(%rbp),%rax
40069c:    89 10                    mov    %edx,(%rax)
40069e:    b8 c0 08 40 00           mov    $0x4008c0,%eax            (line 69)
4006a3:    48 8b 55 f0              mov    -0x10(%rbp),%rdx
4006a7:    48 89 d6                 mov    %rdx,%rsi
4006aa:    48 89 c7                 mov    %rax,%rdi
4006ad:    b8 00 00 00 00           mov    $0x0,%eax
4006b2:    e8 99 fd ff ff           callq  400450 <printf@plt>

A similar instruction is available for the 64-bit wide registers, as you can see if you compiled and disassembled the byte swap example.

Swapping 2-byte words with in-line assembly

To do the same for words containing only two bytes, the processor instruction has to be called by writing a tiny bit of assembly, as an easy built-in function for this is lacking in the GNU C compiler. This small piece of code looks like:

short int *ptr16b, i, num16b = 256*'N'+'P';
*ptr16b = num16b;
printf("Default 2-byte sequence (little endian) : %s\n",array);
__asm__ (  "xchg %b0,%h0\n"  :"=r"(*ptr16b)  : "0"(num16b)  );
printf("Swapped 2-byte sequence (big endian) : %s\n",array);

Again, the memory location to which the pointers refers, is overwritten with a short integer, thus storing two bytes. The resulting output of the first printf() statement is “PNcdefgh” following the little endian definition of the lower value character ‘P’ first. Then, a special compiler option is used to insert assembly code and thereafter the second printf() statement outputs “NPcdefgh”, as the two bytes are first swapped before storing them. The in-line assembly is written using the following format:

asm ( assembler_template : output_operands : input_operands : list_of_clobbered_registers );

The assembler template is a string containing the actual assembly code, while the output operands, input operands and list of clobbered registers are optional parameters. So, how does it work? We’ll use the micro-instruction xchg %bx, %hx which swaps the higher and lower byte of register x. In practice this can be any of the available registers inside the microprocessor. To enable the compiler look for the most suited one,  we’ll use the output operand %0 and let the compiler decide which register will actually be used. With the constraint “=r” the compiler knows that only a register (not a memory location) may be used, and that its previous content is overwritten. A similar operand %1 should be assigned for input of this assembly code. However, as the result of the instruction is stored in the same register it acts on, we need to ensure the compiler loads this same one with the data first. This effect can activated by using the “0″ matching constraint.

Comparison with several shift actions

Which method is faster, assembly or pure C/C++ ? To examine if this micro code optimization is really effective, a small test program is written. In this program two functional similar functions are described, which however differ in implementation:

void swap64_array_c_lang (long long int *start, int number){
long long int data, *end = start + number;
while(start < end){
*start = ( *start >> 56 ) |
(( *start << 40 ) & 0x00FF000000000000) |
(( *start << 24 ) & 0x0000FF0000000000) |
(( *start <<  8 ) & 0x000000FF00000000) |
(( *start >>  8 ) & 0x00000000FF000000) |
(( *start >> 24 ) & 0x0000000000FF0000) |
(( *start >> 40 ) & 0x000000000000FF00) |
( *start << 56 );
start++;
}
}

void swap64_array_asm (long long int *start, int number){
long long int *end = start + number;
while(start < end){
*start = __builtin_bswap64 (*start);
start++;
}
}

A typical solution to solve the byte swapping is C/C++ is a calling the shift operator << to perform part of the swapping while masking other bits using a logical & operator. The disadvantage of this is the overhead of writing out manually all eight byte movements and then combining them. This is also visible in the size of the disassembled code: the pure C language function occupies 235 memory places, while the assembly function call only takes 66 bytes: a memory reduction of 72% (without calling the optimization option of the compiler).

The speed improvement is tested by swapping an array of thousand 64-bit long integers (equal to 8000 bytes) one million and one times: using an odd number enables printing a proof in between both test conditions. By requesting the clock ticks just before starting this for() loop and immediately after, a reasonable accurate result can be obtained: the shift operations in C took 11.83 seconds on a AMD Athlon 64 X2 Dual Core 6000+ processor  while the assembly implementation only needs 4.39 seconds. Thus 2.695 times faster…

Using the compiler option -O3 for maximum performance, the duration of the for() loop with bit shift operations is reduced to 3.96 seconds, while the assembly implementation is even more increased in speed and only needs one second.

Why not using the network-byte-order swap conversion?

Another usable method to switch from little to big endian and reverse, if required by the architecture, is using the network to host short function ntohs() and its counterparts. These functions, for two and four bytes sized words, are present when using the following pre-processing statement: #include <netinet/in.h>. At first glance, it looks like there are two disadvantages: the eight byte long word conversion is missing and these functions do a call to another place in the memory which will cost much more time. The latter disadvantage will be solved by using the compiler option -O3, as this function call will be converter to and in-line set of assembly codes, avoiding a (long) jump in memory. Usage of the network-to-host-conversion functions however has a very big advantage when writing software for both big and small endian machines: these function call will swap the bytes if the host is small endian (as the network is supposed to be big endian) and will do nothing if the host machine is also big endian: the ultimate saving of cpu-cycles.

Improving a real world application: klayout

Although the speed gain looks impressive, the example is very artificial. To demonstrate the actual improvement, I’ve downloaded the version 0.21.14 of the source code of klayout. To measure accurately the speed improvement, also the Ruby language support is needed: klayout offers the possibility to run a script without starting the GUI. To get Ruby (on an Ubuntu system) type the following:

sudo apt-get install ruby ruby-dev

Next, compile klayout including the link to the Ruby headers:

./build.sh -rblib /usr/lib/libruby1.8.so.1.8 -rbinc /usr/lib/ruby/1.8/x86_64-linux/

If all this was done successfully, we now need a small Ruby test script, ByteSwapSpeedTest.rb, which loads a example GDSII file, and exits klayout normally:

app = RBA::Application.instance
mw = app.main_window
mw.load_layout( "/home/peter/example.gds", 0)
mw.exit

To test this script in non-GUI mode type the following (an explicit path to the time command is required in the Bourne Again Shell ‘bash’, see the manual page of ‘time’):

/usr/bin/time -f '%E real,%U user,%S sys' ./klayout -z -r ByteSwapSpeedTest.rb

With an example file of 3.6GByte this takes on average 60.38 seconds. This large time constant avoids relative large fluctuations due to the cache of the CPU or the behavior of the virtual file system.

Now let’s change some code: in the source directory two files perform reading a GDSII stream: dbGDS2ReaderBase.cc and dbGDS2Reader.cc. I’ve modified them using the ntohs() and ntohl() functions as also re-casting some character arrays pointers to (short) integer pointers. You can find the modified code here. Re-compiling it, the size of the binary is now reduced by 4Kbyte: not impressive compared to original size of 23MByte, but saving some memory by using another (and cleaner) interpretation makes sense. The speed improvement of reading the large GDSII file is 2.1%. Hmm, not much faster indeed. To exclude the random variation of starting a process both binaries were tested 14 times, so possibly the improvement can be a little bit more (or less?).

All Debian based Linux distributions (Debian, Ubuntu, Mint) use the same Debian packaging method for distributing and updating software released by the community. Although many open source software packages are already handled by this method, still enough interesting software is currently (still) out-of-scope. With an interest in using open source Electronic Design Automation (EDA) software tools in a professional environment, I’ll try to create my first Debian package.

What is a Debian package? It’s simple an archive, created by the unix command ar, and consists of three main files: debian-binary, control.tar.gz and data.tar.gz. A brief descriptions:

• debian-binary: just a two bytes sized file containing the version number , a single ’2′ character, followed a typical C-style end-of-string character ‘\0′.
• control.tar.gz: contains a file control which describes what should be done to install or remove this package, and a checksum verification file md5sums using the  md5sum algorithm, and optionally pre- and post-installation of removal control files called preinst, postinst, prerm and postrm.
• data.tar.gz: all executables, source and data files, templates etc. related to the package are stored in this file. The content of it should follow the Filesystem Hierarchy Standard (FHS) to comply to the rules of the Debian package method.

The software I want to package is the layout viewer (and editor!) klayout of a.o. GDSII data files. Especially the XOR function to test the difference between to two GDSII files makes it very interesting for comparing layouts and/or debugging the design flow.

The control files

The most important control file control contains information regarding the package’s name (of course), the CPU architecture for which it is built, the maintainers name and e-mail address, and last but certainly not least the dependency of required software libraries. Debian uses a nice relative specified install mechanism, see chapter 7 of the manual. The program klayout needs e.g. the Qt widget library, as also the accompanying XML parser, the (de-)compression library routines and the standard C++ library. The required versions of these packages are simple specified between parentheses in a relational description: as an example: libabc (>=x.y.z) means it requires a package containing library libabc of version x.y.z or later. For klayout the dependencies are written in the control file, in this human readable syntax, as follows:

Depends: libqtgui4 (>= 4:4.2.3), libqt4-xml (>= 4.6.2), zlib1g (>= 1:1.2.3), libc6 (>= 2.11.1)

The version number of the package follows the original package version number, 0.21.14 klayout,  however an minus separator is added followed by the version of this package itself, in total  0.21.14-1. Future patches of this release can than be numberer as 0.21.14-2, 0.21.14-3.   Furthermore, a brief description is needed in the control file, so the end-user knows what kind of program (s)he is trying to install. Three lines will suffice to decide ‘Yes, I want to use this!’ or ‘No way!’.

The data files

As mentioned before the data files should follow the FHS standard to guarantee a consistent directory layout of documentation, executables etc. even when combining several unrelated packages. The binary executable of klayout needs to be placed in the /usr/bin directory to allow all (non-root) users of the system to use the program. This binary executable can be build in the normal way, see here. After doing this symbol table needs to be removed using the following command:

strip ./usr/bin/klayout

By doing this the size of the executable shrinks from 23032 to 19574kByte, saving 15% storage space. Apart from starting klayout direct from the command line, one would also like to have an (sub)entry in one of the pull down menus. Therefore a brief description in the newly created file /usr/share/applications/klayout.desktop is needed. The name of the program, a small comment, references to the executable and icon’s name are added, as also a base and subclass categories are written in this file using the freedesktop standard:

[Desktop Entry]
Version=1.0
Name=Klayout, viewer and editor of mask layouts.
GenericName=layout viewer
Comment=Klayout is a viewer (and editor) of mask layout in a.o. GDSII and CIF format.
Exec=klayout
Icon=klayout
Type=Application
Categories=Development;Engineering;Electronics;

Accompanying with the text of the pull down menu’s entry box a small picture can be placed. This one will be recognized by its name (without suffix) is placed in the /usr/share/pixmaps folder as klayout.png. The results look like this:

Nice, isn’t it?

As the software is released under the Gnu Public License (GPL) a copyright file is also required in the /usr/share/doc/klayout directory, alongside two change-logs: one of the source code itself and one of the Debian package. The first can be copied from the release notes on the website of klayout, the second one, has to be maintained in the future by the package maintainer, me. The directory layout before packing and compressing it, now looks like:

./control
./usr/share/doc/klayout/copyright
./usr/share/doc/klayout/changelog.Debian
./usr/share/doc/klayout/changelog
./usr/share/pixmaps/klayout.png
./usr/share/applications/klayout.desktop
./usr/bin/klayout
./postrm
./postinst
./debian-binary
./md5sums


Packing and compressing

With all content present, we can start zipping the change-log files first, while using the best available compression method of gzip:

gzip --best usr/share/doc/klayout/changelog
gzip --best usr/share/doc/klayout/changelog.Debian

To guarantee correct transfer it makes sense to generate a md5sum checksum of each file. This can be done by the following one-liner, using the most complex Unix command find (so if you succeed in understanding it, you’re certainly not a novice anymore )

find usr -type f -exec md5sum {} \; > md5sums

To avoid static paths in the tar-file make sure to add “./” at beginning of path name:

fakeroot tar -cvf data.tar ./usr
gzip data.tar

The command fakeroot is used to strip user ownership and rights from the file properties so it looks like full root access inside the tar file. In a similar way we can make the control.tar.gz file by typing:

fakeroot tar -cvf control.tar control md5sums postinst postrm
gzip control.tar

Finally, apply the ar command to add, in correct following order, the Debian version file first, control information as second and the data as third file:


fakeroot ar cr klayout_0.21.14-1_i386.deb debian-binary control.tar.gz data.tar.gz


Verification

To verify the packaging is done correctly, the program lintian is called:

lintian klayout_0.21.14-1_i386.deb

The result of this verification look like:

W: klayout: new-package-should-close-itp-bug
W: klayout: binary-without-manpage usr/bin/klayout
W: klayout: postinst-has-useless-call-to-update-menus
W: klayout: postrm-has-useless-call-to-update-menus

No real problems, only warnings left; all well understood. The files postinst and postrm were copied from an example tkdiff package, possibly these triggers are not required for all kind of desktop icons, as I assumed? Also the Intent-To-Package bug (ITP), will do no harm but reminds me of the need of finding an Debian promotor for this klayout package. And I should also spent some time on writing my first man page )

The final result in 32-bit (i386) format is located here, the 64-bit (amd64) version here. The latest version is 0.21.18, here, also as 64-bit (amd64) version here