Archive for the ‘Linux’ Category

Customizing svn diff

26/10/2013

A few years ago I wrote about how my discovery of colordiff had improved my svn diff experience. Since then I’ve made a number of other tweaks to better customize my diffing:

#!/bin/sh -u
exclude_file=/path/to/exclude/file
svn diff --no-diff-deleted --diff-cmd diff -x "--ignore-all-space --text --unified=0" $* | \
  filterdiff --verbose --exclude-from-file ${exclude_file} ${tmpfile} | \
  grep -v --file=${exclude_file} | colordiff | less -RS

I never want to see full diffs for deleted files, hence I give the --no-diff-deleted option to svn diff. (I would also use --no-diff-added if it existed!)

I also regularly diff artifacts archived from automatic jobs which build and test NAG Fortran and C Libraries. There are some differences there that, although they need to be archived for possible future reference, I never need to see on a day-to-day basis — diffs for non-repeatable RNGs, for example. I use filterdiff (from patchutils) to remove these from the svn diff output. I have the patterns to exclude listed in a separate file (pointed to by the exclude_file in the script). These patterns are shell wildcards, so look something like *g05kg*e.x for ignoring the results differences from the Fortran (respectively C) example program for the NAG non-repeatable RNG initializer g05kgf (respectively g05kgc).

A block of differences excluded by filterdiff leaves behind the separator Index: filename from svn diff, so in the script above a call to grep filters these out too. Unfortunately that means each exclusion needs to appear twice in the exclude_file: once as a shell wildcard for filterdiff and then once as a basic regular expression for grep: so as both *g05kg*e.x and g05kg.e\.x say. I haven’t yet worked out how to unify this. You also end up with orphaned ==== separator lines as well, but I don’t feel too inconvenienced by this: I just jump between the resulting diffs according to the presence of the ^Index separator for the diffs that haven’t been excluded. These diffs will come out colorized via colordiff and less -R as discussed in the older post.

Note also that Index lines for deleted files, filtered by svn diff --no-diff-deleted, still appear in the final output. I don’t exclude these at the grep step because I still like to see that a file has been deleted, without needing to see what has been deleted. And although the facility exists in the [helpers] section of ~/.subversion/config to set the script above to override Subversion’s diff implementation, I like to run my diffs by invoking this script explicitly and then I can still use plain svn diff as a sanity check or fallback, especially if filtering is not required or whitespace in the diff is significant.

Source code: https://github.com/matcross/blog/tree/master/customizing-svn-diff

Advertisements

How to set up Fedora for work

14/07/2012

I got a new desktop machine at work. Our helpful sysadmin installed Fedora 17 64-bit for me (as a NIS client and all that jazz). This post is a note-to-self about the additional configuration I had to do to finish getting it ready.

I gave myself sudo privileges: as the local admin user who was added during the install,

sudo visudo
# Added me ALL=(ALL) ALL to the who-what-which section

(Reminder: <ESC>:wq saves and quits in vi! I always forget that syntax…)

Alternatively I guess I could have added myself to the wheel group…

I often need to build 32-bit code, and from this 64-bit environment with gcc -m32 I saw

/usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory

To resolve that I needed to install glibc-devel.i686.

I ran into other missing 32-bit components too, namely

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/4.7.0/libgcc_s.so when searching for -lgcc_s

which, it turns out, is a somewhat-unhelpful way of saying that I don’t have the 32-bit libgcc package installed. yum provides was helpful here:

sudo yum provides "libgcc_s*"
...
sudo yum install libgcc.i686

Also, when doing a link using -static -lm -lutil, I got other terse messages

#/usr/bin/ld: cannot find -lm
#/usr/bin/ld: cannot find -lutil
#/usr/bin/ld: cannot find -lc

because I didn’t have glibc-static installed.

Other extra packages and setup I needed were:

  • colordiff: I like to have colorised svn diffs;
  • Jenkins. The documented installation instructions are good

    sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
    sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
    sudo yum install jenkins

    That gave me a sandbox installation to mess with;
  • sudo gnome-control-center to set VPN routing (using the IPv4 tab) to access the new machine when I’m outside the work firewall. The resulting configuration is written to /etc/sysconfig/network-scripts/route-Wired_connection_1;
  • edited /etc/exports to export my home drive on the machine to the work network (/mydir *.workdomain.co.uk(rw,insecure,async)) and to the VPN IP. Installed nfs-utils and enabled (at boot) and started the mount daemon to complete the export

    sudo yum install nfs-utils
    sudo systemctl enable nfs-mountd.service
    sudo systemctl start nfs-mountd.service
  • additional Gnome configuration:

    sudo yum install gnome-tweak-tool gnome-font-viewer gnome-shell-extension-alternative-status-menu gnome-shell-extension-remove-accessibility-icon gnome-shell-extension-remove-bluetooth-icon gnome-shell-extension-dock gnome-shell-extension-alternate-tab
  • enabled RPM Fusion free and nonfree repositories using RPM Fusion Configuration and enabled MP3 support in Rhythmbox by installing gstreamer-plugins-ugly;
  • installed dependencies for Spark IM Client

    sudo yum install libX11.i686 libXext.i686 libXi.i686 libXp.i686 libXtst.i686 alsa-lib.i686 unixODBC.i686 compat-libstdc++-33.i686

    (although the Spark 2.6.3 RPM has a broken dependency on libodbc and libodbcinst);
  • installed python3, rdesktop, libusb-devel, lcov, octave, mpfr-devel, libmpc-devel.

Fedora-ing on a Rainy Afternoon

01/05/2012

The weather last Sunday afternoon was purgatorially dreary. I decided I’d try installing the Fedora 17 beta on my feeble and unmiraculous Dell Optiplex 260 at home in advance of upgrading other more important machines when the full release is available, so I downloaded the full i386 install DVD .iso. (Actually, being accustomed to having a poor broadband connection I had set this to download the night before.)

I don’t have any blank DVDs to hand at home, but I do have a nice 16GB Sandisk Cruiser Blade USB stick that I wanted to try installing from. (That’s bound to be kinder on the environment too, right?!?)

There is a good page on the Fedora wiki covering how to create and use Live USB that seemed relevant to this exercise. However, I made the initial mistake of only reading half the page and blindly followed the instructions to make a Live USB image using the LiveUSB Creator.

After RTFW (Reading The Fedora Wiki) a little more, I saw that these weren’t the instruction droids I was looking for; I wanted how to make a bootable USB drive to install Fedora instead of using a physical DVD, further down the page. I ran this advice without a hitch.

Nearly done? Well, I’d forgotten that I couldn’t boot the OptiPlex from USB (and began dimly remembering this rigmarole from Fedora-16 time). It was still dank outside, so I had a look around the web and found some great instructions from Dell which made me realise that if I’d bothered to update my BIOS since 2005 I would have support for booting from USB. Oops.

With the BIOS upgraded and the USB drive inserted I was given the option in the boot menu to boot from USB, and it worked, and the installation (well, upgrade really), seemed to go fine, and here I am writing up these notes using Fedora 17 Beta. Ta-da!

Fedora 17 Wallpaper

I’m pretty certain I used to be able to take screenshots differently with Fedora 16 though, and there are some odd messages when I try to shutdown, … Let’s report some bugs!

Dear disk – don’t die

30/03/2012

My trusty crusty home desktop (a Dell OptiPlex GX260) is nearly ten years old, and it’s been a bit flaky on recent occasions when booting up Fedora 16. One time this week I let it have about six attempts before I gave up; it would get stuck – after quite a while – and say something like

udevadm settle - timeout of 120 seconds reached, the event queue contains:
/sys/devices/pci0001:00/0001:00:02.0/0001:01:0b.1/usb3/3-2 (623)
/sys/devices/pci0001:00/0001:00:02.0/0001:01:0b.1/usb3/3-2/3-2:1.0 (624)
/sys/devices/pci0001:00/0001:00:02.0/0001:01:0b.1/usb3/3-2/3-2:1.0/0003:05AC:1000.0001 (625)
/sys/devices/pci0001:00/0001:00:02.0/0001:01:0b.1/usb3/3-2/3-2:1.0/0003:05AC:1000.0001/hidraw/hidraw0 (626)

(Unfortunately I don’t have the exact details to hand. I’m writing this from memory, but the OP’s question at udev reaching timeout is pretty much what was happening to me.)

The Disk Utility in the Accessories area

Accessories, Disk Utility

told me in the SMART Data section for the drive that there are bad sectors

SMART Data, bad sectors

For any repair work to be able to happen the drive has to be not mounted, so I booted from an old Fedora 15 Beta LiveCD I had lying around. I ran

$ sudo lvdisplay

to find my root partition (since it’s on a logical volume), and then ran

$ e2fsck -c my_root_partition

After several days of usage following this, booting seems a lot more stable. Hopefully there are another few years left in the old girl.

libc ya later, alligator

28/03/2012

Yesterday, like a n00by n00b, I deleted my Fedora’s /lib64/libc.so.6 symbolic link. It was intentional; not an act of deliberate vandalism, but neither in retrospect do I think it would have actually helped me achieve what I was trying to. The details of what I was attempting aren’t important. What shocked me is
how destructive this simple mistake was
: remember – I’ve only deleted the symlink, not the actual target library.

Try it yourself!

$ sudo rm /lib64/libc.so.6
$ ls
ls: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory

It’s all gone wrong. You can get some functionality back by setting LD_PRELOAD, e.g.

$ setenv LD_PRELOAD /lib64/libc-2.13.90.so
$ ls
(Stuff)

Yay. Ish: we really want to restore that little symlink though.

$ su
su: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory

I guess that LD_PRELOAD trick isn’t supposed to work here with stuff like su, otherwise we could load in any old junk library to bypass security, etc.

Is there really no way this can be fixed while the system is still live?

The superuser question How to restore /lib/libc.so.6? helped with the above, but I was hoping to avoid having to continue with the advice there which is to restore using a LiveCD. The problem has such a tantalizingly simple cause I was rooting for a simple solution.

In the end I dug out an old installation CD, but not first without seeing if the system would reboot. Ha!

Kernel panic - not syncing: Attempt to kill init!

and so on and so forth. But in the end sorting the problem out with the installation CD was simple. The rescue option mounted my installation under /mnt/sysimage/, so I just

$ cd /mnt/sysimage/lib64
$ ln -s libc-2.13.90.so libc.so.6

and with a reboot again, everything is back to normal.