Cobra's bits (Posts about latex)

latexdiff-vc

Cobra — Mon, 03 Oct 2022 12:47:34 GMT

Since my first post on version control systems (VCS) and latexdiff, the whole kaboodle has become much easier for the user. Instead of looking for some python scripts to ease the generation of diffs, latexdiff itself offers this functionality. For example,

latexdiff-vc --hg --pdf -r 0 example.tex

will create a diff between the original(-r 0) and current version of example.tex handled by mercurial (--hg), and compile the resulting diff by pdflatex (--pdf).

Even better, latexdiff supports multi-file documents, the use of which is highly advisable for dissertations and books or other long documents. The workflow is simple: as usual, we create the repository and add files to it, but this time all files have to be included:

hg init
hg add master.tex file1.tex ... filei.tex ... filen.tex
hg commit -m "Initial version"

And after editing an arbitrary file:

hg commit -m "Did this or that"
latexdiff-vc --hg --pdf --flatten -r 0 master.tex

Voilà, here's your compiled diff. 😎

As a sidenote: I have seen way too many students trying to compile the active chapter of their dissertation, just to realize that only the master file including their active chapter can be compiled. Hence, they classical sequence: compile – oops, switch tabs, compile, switch tabs, edit, compile – oops, etc. etc. ad nauseum. Except for masochistically inclined witers, there's a simpler and better way.

Indeed, many specialized LaTeX editors (TeXWorks, TeXStudio) and also several of the most prominent general programming editors with a LaTeX extension (Sublime Text with LaTeX Tools, VS Code with LaTeX Workshop, Vim with Vimtex ...) respect the following magic command at the top of the active chapter:

% !TeX root = master.tex

Only Emacs with auctex needs a slightly different syntax:

% -*- TeX-master: "master.tex" -*-

After setting this command, you can just type and compile, and your focus will not be distracted any longer by the compilation cycle sketched above.

Modern times

Cobra — Sat, 05 Sep 2020 11:52:22 GMT

Since the beginning of time (or so it seems), I've had Emacs installed with the extension AUCTeX to handle LaTeX documents. And, mind you, I'm still using it from time to time! As a matter of fact, in 2018 I've worked on a number of manuscripts exclusively with Emacs to prepare myself for the editor shootout I've promised for the end of 2018 (and which may or may not be done by the end of this year). I'm still quite happy with it, that much I can say already now.

Perhaps you can then understand my surprise when yay told me that AUCTeX has been orphaned on the AUR. I was even more surprised when I saw that the maintainer was Stefan Husmann, who is also the maintainer of several hundred other packages and a moderator on the German Archlinux forum. Not the guy to thoughtlessly abandon a package on a mere whim.

And then it hit me: of course! Emacs has got it's own package manager (ELPA) some time, well, perhaps two years ... actually, eight years ago. 😣

So what's the meaning of this post? Let's say that there're one billion computer users out there. Only one percent of these know what an editor is, and only one percent of these again are actually using one. Of these again, only one percent use Emacs. Once again one percent of these Emacs users use AUCTeX, but more than 80% of these guys have installed AUCTeX via ELPA, the recommended and canonical way. I'm not one of them. Am I the only one? No, if we do the math, it turns out that there's one kindred spirit who is in the same situation like me. This post is for you, my brother in arms!

Well, as stated above, Emacs recommends installing AUCTeX via ELPA. After removing AUCTeX from the AUR, we can install manually

M-x package-install RET
auctex

or, in a properly maintained init.el (like mine), automatically:

(require 'package)

(add-to-list 'package-archives
        '("melpa" . "`https://melpa.org/packages/" <https://melpa.org/packages/">`_))

(package-initialize)
(when (not package-archive-contents)
        (package-refresh-contents))

(defvar myPackages
   '(better-defaults
         material-theme
         ein
         elpy
         flycheck
         py-autopep8
         ac-math
         auctex
         auto-complete-auctex
         ))

(mapc #'(lambda (package)
        (unless (package-installed-p package)
          (package-install package)))
          myPackages)

and

;;(load "auctex.el" nil t t)
;;(load "preview-latex.el" nil t t)

While I was pondering the question whether this post would be relevant for anybody at all, I found these news: Levee, a vi clone, has got a new major release after 30 years. Now that's the spirit! Compared to the estimated number of users interested in this update (interestingly, the only comment in the AUR is from Stefan Husmann), my post is for the masses. To celebrate this Chucknorishness of software development, I've installed levee and prepared this text in it. It was ok (just like vi), but David Parsons will certainly understand if I say that I prefer vim for everyday work.

One does not simply exit vim

Cobra — Fri, 26 May 2017 12:06:37 GMT

A few days ago, stackoverflow has hit a major milestone: the community has helped one million developers to exit vim. If that isn't reason for celebration, nothing is.

'Of course, these aren't real programmers, since those use something else entirely.'

Surely the mighty ed?

'Not even close.'

For those still interested in the grandfather of vi (aka the most user-hostile editor ever created): here's an excellent little tutorial. By the way, you can exit ed by the canonical quit command of Unix applications: q. Much more intuitive than vim!

I'll write about the editors I'm using (and why) in the near future.

LaTeX vs. Unicode

Cobra — Sun, 13 Nov 2016 10:03:52 GMT

I'm using matplotlib to create figures for my publications. For axes labels, legends, and everything else requiring text and symbols in a figure, I've so far used the excellent LaTeX support of matplotlib, and the results are (obviously) highly satisfactory:

There's a disadvantage, though: there are not too many fonts to chose from. Naively, I thought that this limitation would be lifted if I wouldn't use LaTeX, but Unicode instead:

And wouldn't XeLaTeX even combine the advantages of both?

As you can see, matplotlib allows you to use any of these options, but what you don't see is that the desired results can be achieved only with a very limited set of fonts. For example, there are only a few fonts that include the unicode character for a 'superscript minus' (for an overview, see here). Sadly, most of these are part of the ClearType Font Collection, which was introduced by Microsoft with Windows Vista. Free fonts with a 'superscript minus' include Dejavu Sans, Free Sans, and Free Serif. If the 'superscript minus' is included instead as a command by employing the internal LaTeX support of matplotlib, many more fonts become accessible. Examples are shown in the table below. But even then one can't make any assumptions: while Source Sans Pro works fine, Source Serif Pro doesn't. I have no idea why.

You see from my last statement that this post in not in the least authoritative. I'm just toddling around, and if you find a better way, I'd appreciate corrections and additions. That's particularly true for the case of XeLaTeX, the use of which seems to require OTF-only fonts with math table support. I wasn't even able to find a single Sans Serif font with this profile 😞 . Others have similar problems.

Renderer	Serif	Sans Serif
LaTeX	Palatino, Fourier	Kurier, CM Bright
Unicode	Noto, Gentium Plus	Open Sans, Source Sans Pro
XeLaTeX	Libertinus, XITS	?

Finally, here's an archive containing the three scripts I've used to create the figures above. In each case, I let matplotlib render a pdf, convert that into an svg by pdftocairo, and compress this svg files by gzip:

./plot_uc.py
pdftocairo -svg plot_uc.pdf plot_uc.svg
gzip -S z plot_uc.svg

The results are compressed scalable vector graphics that are fully compatible with inkscape if a post-processing should be necessary. That's how I got the unicode logo in, by the way. 😉

In the old days

Cobra — Sat, 03 Sep 2016 12:43:46 GMT

Research places like the one I'm associated with are characterized by a constant turnover of personnel. Currently, our 20 senior scientists are supported by about 50 assistant and associate researchers on temporary positions. Our annual turnover rate is thus as high as 30 to 40%, meaning that I meet about 15 to 20 new people every year.

The level of understanding in physics and material science fluctuates, but does not seem to deteriorate over the years. That's the good news. What does decline significantly is the ability to read and write and to use a computer efficiently. At the same time, the fraction of people with an undue sense of entitlement is growing dramatically.

What do I call an “undue sense of entitlement”? Well, imagine. It's your first day at a new place where you hope to perform top-notch research yielding results important enough to publish them in prestigious journals. You are shown into your office, which you share with some more experienced colleagues, and on your desk sits a brand-new 24 inch Full-HD display connected to an equally brand-new desktop computer. And exactly that's the moment when you demand, loud and clear, two monitors. The bigger the better! And an hour later, you call our IT service to demand the real Office. And Photoshop! When the IT freaks ask for which purpose you need this software, you most strongly express your righteous indignation. First of all, that's none of his business, and second, that should be obvious! After all, you have letters to write. And later, there may be images whose contrast needs to be increased.

Now, that's exactly what you would do, right?

No, of course not. No halfway sensible person would behave in this way. Alas, every year we get more and more young people with this attitude. Experience tells us that the people demanding the most are the ones returning the least. They also tend to create constant trouble: they are more concerned with their own self-importance than with their research, and are generally ignorant, obnoxious, and unproductive.

I still vividly remember my own time as a PhD student at the MPI-FKF. I had previously written my diploma thesis on an HP Vectra, an IBM AT compatible, which I had to share with the six or seven members of our research group. At the weekends, I also had access to my own computer, an Schneider PC1512 that I primarily used for running Pascal programs. God, did it feel slow compared to the Vectra at work! At the MPI, however, computing was not yet “personal”. Instead, VT220 text terminals were offered for the interaction with the VAX station in the basement. I'd say we had perhaps 20 terminals for about 300 scientists.

Basically, these terminals were used for three essential tasks. First, one could receive and send e-mails using the 'mail' program. That was my first contact with the Internet and I found it exciting! Second, we wrote our manuscripts using EVE, the extensible versatile editor, compiled them with LaTeX, converted the resulting dvi file to postscript with dvips, and used lpr to send the postscript output to one of the three or four Hewlett-Packard LaserJets located at strategic locations (mine was close to the beer machine¹) in the institute. The VT220 could not display any graphics, so we had to print to see if our LaTeX code did what we wanted! Third, EVE was also used to write FORTRAN code, and there were always a number of computations running on our VAX. I do not remember which model we had, but a VAX 6360 from 1988 had six processors clocked at 16.7 MHz and an estimated performance of 22 MIPS, all for a meager $752K. A high-end smartphone is easily 1000 times as fast.

Just that you have an idea of how it feels to work on a VT220, here's an impression recorded by jstn. The little cube on the left is the VT220 displaying a terminal of the Mac Pro it is connected to.

Between January 1990 and September 1991, I wrote 10 manuscripts on this little cube as the first author (all of which got published) followed by my dissertation which I've submitted just before Christmas 91. How I did that? Instead of complaining, I typed.

¹ Yes, you heard right: we had a beer machine. All right, it was actually a standard soda machine, but it offered two kinds of beer in addition to Coke and Co.: Stuttgarter Hofbräu Pils and Export, if I remember correctly. The machine was heavily frequented and we had the most illuminating discussions in the park just outside the canteen, where the beer machine—and the printer—were located. Many great ideas originated from these discussions. Nowadays, where people wear a helmet when going to the loo, that's unimaginable.

Long-time support

Cobra — Sun, 27 Dec 2015 13:18:19 GMT

LaTeX files with a bibliography (or an index) require several compiler passes to resolve all references for building:

latex paper.tex
bibtex paper.aux
latex paper.tex
latex paper.tex

Tedious, isn't it? For that reason, wrappers have been invented, automating this procedure. The veterans on this field are latexmk and rubber, but there are several others, including a new kid on the block called latexrun that sounds truly promising.

I've used rubber for many years, despite the fact that its development had stalled nine years ago. Rubber has worked for me since scientific publishers are ... ahem ... extremely conservative: they will probably stick to the pdflatex/bibtex workflow for the next few centuries (or forever, whatever comes earlier).

However, for documents not intended for submission to a publisher, I've long ago switched to XeLateX/Biber. For example, my curriculum vitae is to be compiled with these engines rather than with their prehistoric ancestors. Since it is technically identical to its template, you can download Adrien's CV to test the wrapper you want to use.

The following commands should compile this CV in a single run, including the bibliography:

latexmk --xelatex --synctex=1 --silent cv.tex

rubber --module xelatex --module biber --synctex cv.tex

latexrun --latex-cmd xelatex --bibtex-cmd biber cv.tex

In July 2015, only latexmk managed to do that, while rubber 1.2 failed despite my manual addition of XeLaTeX and Biber support. I was only mildly disappointed: after all, rubber had become lame and grey, and I realized that it was time to leave it behind. I was surprised, though, that latexrun, the newly developed contender, failed as well.

Now, to my absolute astonishment, the rubber project suddenly became alive again this year. After nine years of inactivity, rubber 1.3 was published in October, and 1.4 just a few days ago. Rubber 1.4 passed the test above with flying colors, and twice as fast as latexmk. What's more, SyncTeX support, which I previously smuggled in manually, has now been added to the rubber feature list as well.

Rubber thus manages to once again take the crown of LaTeX wrappers, with latexmk just seconds behind (literally) because of its inferior performance. Latexrun in its present state leaves much to be desired, but I'll keep an eye on its further development (if any 😉 ).

Résumés and indices

Cobra — Sun, 30 Mar 2014 16:01:09 GMT

I receive a constant flow of applications for PhD student and postdoc positions. A quick check of my trash folder in kmail reveals that the number of applications over the past 4 month is at least 27. Including the 3 I didn't trash, this number means that I receive about 90 applications per year.

Fortunately, I don't have to spent much time (<10 s) with most of them. Over the years, I established the following policy:

(i) If the application is not send to me, but to unspecified recipients: trash it.
(ii) If the cover letter starts with 'Dear Professor' without explicitly referring to a specific person, or with 'Dear Professor Paul Drude' and thus referring to the wrong guy, or with 'Dear Professor Braudd' or any other sad imitation of my actual name: trash it.
(iii) If the cover letter has no reference to where I work, or what I do: trash it.
(iv) If the cover letter contains several grammatical or typographical errors: trash it.

If the applicant has come that far, I will at least send a brief mail informing him or her that we don't have a position available at the moment. In the unlikely case that we are indeed looking for somebody at this very moment, I'll have a quick (!) look at the attached résumé or curriculum vitae. What I see then is often pathetic — poorly formatted Word files with more grammatical errors and typos than you'd expect even from a mediocre high school student — and rarely informative: Chinese applicants frequently refer to their role as 'model student of the year' and their perfect command of the English language, while their Indian counterparts seem to view their enthusiasm for cricket as essential information.

In some cases, however, I'm pleased by what I see. An astonishingly high number of these outstanding cases are prepared by applicants in German-speaking countries using the 'moderncv' class of LaTeX. This choice not only guarantees a well-structured and aesthetically pleasing résumé, but is also certain to register as a proof that the applicant has no problem dealing with LaTeX (which is definitely a plus in the natural sciences).

The disadvantage is that applications based on moderncv in its default configuration all look the same, and lack any obvious individual characteristics. For me, that's such a serious drawback that I do not use the moderncv class for my own résumé but instead still the one I've improvised 25 years ago based on the article style and doing all formatting with the tabbing environment. Now, I'm not looking for a job, but I still need to attach my résumé to project applications, and at times I'm musing about how nice it would be to start from scratch and base my résumé on a more modern foundation.

Well, at a lazy afternoon I searched the net and came across several interesting templates for a résumé or curriculum vitae. Technically, the template provided by Adrien Friggeri is very attractive: it employs TikZ for the header, XeTeX and fontspec to use an arbitrary otf font, and biblatex/biber to automatically print selected sets of publications. The design is on the edge of being extroverted, but that's easy to change.

I'm currently playing a little with this template, and I'm really enjoying it. In particular, it never fails to impress me how much we can compress information in the digital age. We can put a world into a single link, and I wonder why I don't see that more often in applications from young (and thus 'digitally native') researchers.

Here's one example:

All of the lines following 'about' are active links when viewed in a pdf reader. They show my identity as researcher from different perspectives, but foremost, they list my publications and citations. What could be more important?

Here's another:

Only one link, but an important one. And certainly, I would treat an application offering explicit values of the publication indices very favorably. 😉

What are these indices? Well, they are more or less smart attempts to compress the 'value' of a certain scientist's research into a single number. The most popular and prominent one is the h index, which is criticized for several reasons, among them the fact that it can be obtained from the number of citations (in my case, the agreement is almost perfect). The g index does not have this weakness, but is more difficult to estimate (I use a script for that). Finally, the i10 index has been created by Google in the attempt to promote its Google scholar service. It's determination is trivial and essentially serves to distinguish wheat and chaff.

None of these indices is free from weaknesses, and none of them alone provides a reliable impression on a researcher's performance. Taken together, however, these indices can reveal much.

As an example, let's compare my indices to those of a valued colleague of about the same age: Steven C. Erwin. At the time of writing, his h, g and i10 indices are 37, 79, and 77, respectively. Evidently, the first two of these indices are close to mine, but his i10 index is drastically lower. Hey...that looks good for me, right?

It doesn't. Nearly equal g, but vastly differing i10 indices simply signify that the guy with the lower value for the latter (i.e., Steve) is more efficient: he publishes less trash which is immediately forgotten.

If I would have to chose between Steve and me, I would take him. Fortunately, he's a theorist, and thus not of much use for anything. 😄

Automated LaTeX

Cobra — Sun, 05 Jan 2014 17:08:16 GMT

We've got several tools which intend to automate the generation of complex documents by LaTeX. Without these tools, the generation of a publication ready for submission would require several compiler passes to resolve all references for building, for example, the index and the bibliography. This thread illustrates the problem and highlights some of the most popular of these tools.

For my modest requirements, I've found that rubber is perhaps the most convenient choice. Rubber seems not to be in active development any more, but does what it should in the most unobtrusive way (and support for XeLaTeX is easy to add).

For the command line, I use a simple makefile:

TEXFILE = $(wildcard *.tex)
PDFFILE = $(TEXFILE:.tex=.pdf)
VIEWER = mupdf

all: pdf

pdf: $(PDFFILE)

%.pdf: %.tex
    @rubber --pdf $<
clean:
    @rubber --clean $(TEXFILE:.tex=)
tidy:
    @rubber --clean --pdf $(TEXFILE:.tex=)
view: pdf
    $(VIEWER) $(PDFFILE)

.PHONY: pdf clean tidy all

For vim, I use TeX-PDF (and LaTeX-Box for command completion). TeX-PDF respects my makefile, if one is present, or uses rubber or even plain pdflatex as a fallback.

On my notebook with ArchBang, however, TeX-PDF opened Gimp for viewing the pdf, which is a wonderfully absurd choice.

To correct that, I defined mupdf as the default application for opening pdf files:

xdg-mime default mupdf.desktop application/pdf,

but that didn't change a thing.

I finally found out that this behavior was caused by a glitch in the default configuration of ArchBang:

$ grep DE ~/.xinitrc
export DE=openbox

Openbox is not one of the possible values of the environment variable DE, and is thus not recognized by xdg-open. One should either substitute 'openbox' by 'xfce', or comment out the entire command.

Word compatible

Cobra — Fri, 04 Oct 2013 16:57:58 GMT

Working in a publicly funded research institute instead of a university has many advantages. The most obvious ones are a decent funding and the lack of any teaching obligations. Disadvantages, however, also exist. In particular, we have to prepare an annual report in which we present the main activities to our advisory board, guests and third-party funding agencies.

For the past 20 years, we have prepared and produced our annual report ourselves using LaTeX. The report grew in volume until reading and correcting all contributions became finally unmanageable. We thus decided last year to focus on our most important results. Furthermore, we outsourced the layout of the report to an external media agency to be able to concentrate on the content. We then supplied the content in the form of text files enriched with LaTeX directives and graphics files in postscript format.

The first report produced in this way was indeed quite presentable and left a favorable impression on most people. This year, however, we were told that the media agency "had severe difficulties with converting LaTeX", and were "urged to produce conventional word-compatible files."

"Conventional word-compatible files". There's a whole world of ignorance in this short statement. Evidently, the agency employs people with a good sense for color and arrangement, but no idea about technical issues.

Because "converting LaTeX" is actually straightforward:

pandoc -s source.tex -o result.docx

transforms a standard LaTeX file to OOXML.

Complicated equations may not be converted successfully in this way. In this case, it is better to export to OpenDocument:

pandoc -s source.tex -o result.odt

Now just mark those equations not converted and click on the π-icon of the LibreOffice extension TexMaths. Select svg as format. Then save as docx.

Existing vector graphics is best converted to bitmaps

pdftocairo -png -r 1200 image.pdf image.png

and then imported into the "word-compatible file".

There are no "severe difficulties". That's hipster nonsense.

Collaboration

Cobra — Sun, 24 Feb 2013 14:19:49 GMT

Last week, I had to edit a manuscript of a fellow senior scientist. The file he gave me was named (I'm not joking) “manuscript.tex”. Since I usually work on many manuscripts in parallel, I gave it a unique name, edited it, and returned it as “xy_abc13_1.tex” using my usual naming convention with xy being the author's name and abc the journal's.

After other authors had their say, a second round was started and the revised manuscript came back to me with the name “manuscript.tex”. I discovered that several of my changes had been overwritten by later changes or were reverted to the original. When I confronted the author with this irritating fact, he told me that he cannot be held responsible because it's obviously way too complicated to keep track of the changes by renaming the file.

I don't share this point of view, but it's true that there are simpler ways to keep track of changes than to manually rename files, namely, version control systems.

In the following, I show you how to profitably use the version control systems mercurial or, alternatively, git when preparing a manuscript or thesis, and how to combine this version control with latexdiff, an indispensable tool for visualizing the changes made from one revision to the other. Using version control and latexdiff in conjunction is dead simple and has many benefits.

After installing mercurial (or git), we first need a minimum configuration in form of an .hgrc or .gitconfig file containing the name and e-mail address of the local user.

mercurial (.hgrc):

[ui]
username = John Doe <[john.doe@doe.de](mailto:john.doe@doe.de "john.doe@doe.de")>

git (.gitconfig):

[user]
name = John Doe
email = [john.doe@doe.de](mailto:john.doe@doe.de "john.doe@doe.de")

We can now initiate our first project:

mkdir project
cd project
cp example.tex project/
hg init
hg add example.tex
hg commit -m "Original version."

The syntax of git is precisely the same.

Now let's edit our manuscript "example.tex", save the changes (under the same name!), and commit the first revision:

hg commit -m "First revision."

hg log should show you the existence of two revisions: 0 and 1. Git has a more complicated numbering scheme as you will discover when issuing git log.

You're now using a VCS! Now, that wasn't that difficult, was it?

Let's add two more commands to our repertoire. Suppose you edit the file, but you are unhappy with the changes and actually would like to start afresh. A simple hg revert -r 1 brings you back to the state of revision 1. And if you have already committed the changes? In this case, hg update -r 1 defines revision 1 to be new head from which further development will occur.

All right, we can track changes, but how can we visualize them? With LaTeXdiff, of course:

There are several ways to use Latexdiff in conjunction with a VCS.

For example, Latexdiff itself supports git. latexdiff-vc -r f76228 example.tex generates a tex file marking all changes between the current revision and revision f76228. ~~You have to compile this file yourself to obtain a pdf you can view and print.~~ See update at the end of this post.

Mercurial supports external diff programs, and this fragment in your .hgrc allows you to use latexdiff within hg. hg latexdiff -r 0 example.tex > diff.tex creates the tex file containing all differences as above. Once again, you have to compile this file yourself to obtain a pdf you can view and print.

Far more convenient are scripts which automate this procedure. Pål Ellingsen and Paul Hiemstra wrote python scripts supporting mercurial (and additionally git in the case of Paul's script) which automatically generate the pdf containing all changes, including the additional runs necessary for bibtex, and even supporting multi-file documents such as used for writing theses and books. Create and view the diff by

diffLatex.py -r 0 example.tex

scm-latexdiff 0:example.tex.

Paul's script also allows us to view the differences between arbitrary revisions: scm-latexdiff 0:example.tex 1:example.tex.

I still give files meaningful names, such as “ob_prb13_1.tex” for my first Physical Review B in 2013. But I will never again worry about file version numbers. What's more, the above scheme is convenient, foolproof (let's see) and entirely transparent. I plan use it for all of my projects in 2013 and the years to come. If only I could convince people to do the same ...

Update 29.07.22:

Recent versions of latexdiff also support explicitely selecting the version control system and compiling the diff file, generating, for example, pdf output:

    latexdiff-vc --hg --pdf -r 0 example.tex

See man latexdiff-vc for further options.