Etiquetas
emacs, latex, orgmode, R, reproducible research, visualization
This post provides technical details about the making of my book “Displaying time series, spatial, and space-time data with R”, a fully reproducible book including hundreds of code chunks and graphics. Hopefully this post will help others when writing a reproducible document.
Tools
Among the tools that can create reproducible documents with R, I decided to use these gems of open-source software:
- GNU Emacs as development environment.
- R (of course!) with Emacs Speaks Statistics.
- org-mode for authoring text and code.
- LaTeX with AUCTeX to produce the final document.
Each chapter is a separate orgmode file
I use a different orgmode file for each chapter. Each file begins with a simple header. The first line uses #+PROPERTY
to specify buffer-wide header arguments (it sets session to R
). The second line uses #+OPTIONS
to define export settings (^:nil
disables TeX-like syntax for sub- and superscripts).
#+PROPERTY: header-args :session *R* #+OPTIONS: ^:nil
When I finished the code and figures in a chapter, but I need to make minor changes and corrections in the text, I need to disable code evaluation. Then the first line of the header is
#+PROPERTY: header-args :session *R* :eval no-export
An example of an orgmode file follows. You have to export it with C-c C-e C-b l l
. With C-c C-e
the export dispatcher is started and l l
chooses the LaTeX exporter and produces a file. C-b
sets body-only on, and the LaTeX file will only include the body without headers.
The content of the file shows rendered by GitHub. Click here for the raw content.
<sec:sectionA>
Code is enclosed within #+begin_src/#+end_src
blocks.
x <- 1:10
y <- 1:10
These blocks are extracted creating a source code file (tangling)
with C-c C-v t
. Without header arguments the code chunk is exported
and tangled as is. Use C-c
’ to edit the current code block. This
brings up a language major-mode edit buffer containing the body of the
code block. Use C-c
’ again to exit. If you need to evaluate it use
C-c C-c
.
Using the header argument :exports none
the next code chunk is not
included in the exported file (LaTeX or HTML) but is included in the
tangled file (R
code). This option is useful to add configuration
code that you don’t want to show in the book.
Use :tangle no
if you need to exclude a code chunk from the tangled
file.
Use C-c C-l
or [[]]
to insert a link or a reference: Section
sec:sectionA.
Figure fig:dummyGraphic is exported to a PDF file using the header arguments
:results output graphics
. The results header argument writes the
graphics output to the file specified in the =:file= header
argument. You need this specification when usinglattice
and
ggplot2
graphics.- =:exports both=. Both the code and results are included in the LaTeX
exported file.
xyplot(x ~ y)
The next lines insert the PDF file with includegraphics
inside a
float figure in the LaTeX file. Use #+CAPTION
to define the caption
and #+ATTR_LATEX# to define the includegraphics
arguments.
The exported LaTeX file is here:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
\section{Code blocks} | |
\label{sec-1} | |
\label{sec:sectionA} | |
Code is enclosed within \texttt{\#+begin\_src/\#+end\_src} | |
\href{http://orgmode.org/manual/Structure-of-code-blocks.html}{blocks}. | |
\lstset{language=R,numbers=none} | |
\begin{lstlisting} | |
x <- 1:10 | |
y <- 1:10 | |
\end{lstlisting} | |
These blocks are \href{http://orgmode.org/manual/Extracting-source-code.html}{extracted creating a source code file} (\emph{tangling}) | |
with \texttt{C-c C-v t}. Without header arguments the code chunk is exported | |
and tangled as is. Use =C-c `= to edit the current code block. This | |
brings up a language major-mode edit buffer containing the body of the | |
code block. Use \texttt{C-c '= again to exit. If you need to evaluate it use | |
=C-c C-c}. | |
\subsection{Header arguments} | |
\label{sec-1-1} | |
Using the \href{http://orgmode.org/manual/Specific-header-arguments.html#Specific-header-arguments}{header argument} \texttt{:exports none} the next code chunk is \textbf{not} | |
included in the exported file (\LaTeX{} or HTML) but is included in the | |
tangled file (\texttt{R}). This option is useful to add configuration | |
code that you don't to show in the book. | |
Use \texttt{:tangle no} if you need to exclude a code chunk from the tangled | |
file. | |
\section{References} | |
\label{sec-2} | |
Use \texttt{C-c C-l} or \texttt{[[]]} to insert a link or a reference: Section | |
\ref{sec:sectionA}. | |
\section{Figures} | |
\label{sec-3} | |
Figure \ref{fig:dummyGraphic} is exported to a PDF file using the header arguments | |
\begin{itemize} | |
\item \texttt{:results output graphics}. The \href{http://orgmode.org/manual/results.html#results}{results header argument} writes the | |
graphics output to the file specified in the \href{http://orgmode.org/manual/file.html#file}{\texttt{:file} header | |
argument}. You need this specification when using \texttt{lattice} and | |
\texttt{ggplot2} graphics. | |
\item \href{http://orgmode.org/manual/exports.html#exports}{\texttt{:exports both}}. Both the code and results are included in the \LaTeX{} | |
exported file. | |
\end{itemize} | |
\lstset{language=R,numbers=none} | |
\begin{lstlisting} | |
xyplot(x ~ y) | |
\end{lstlisting} | |
\includegraphics[width=.9\linewidth]{myFig.pdf} | |
\subsection{\LaTeX{} attributes} | |
\label{sec-3-1} | |
The next lines insert the PDF file with \texttt{includegraphics} inside a | |
float figure in the \LaTeX{} file. Use \texttt{\#+CAPTION} to define the caption | |
and \#+ATTR\_LATEX\# to define the \texttt{includegraphics} arguments. | |
\begin{figure}[htb] | |
\centering | |
\includegraphics[height=0.45\textheight]{myFig.pdf} | |
\caption{\label{fig:dummyGraphic}A dummy graphic} | |
\end{figure} |
A LaTeX file for each Part
Each of these exported files is a children of a LaTeX file with a very simple structure. It starts with \part
and includes several \chapter
. Under each \chapter
you will find an \input
statement that imports the LaTeX file produced with our orgmode file into its parent.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
\part{My First Part} | |
\label{part:part1} | |
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam nec | |
lacus nec dui semper aliquet sit amet in leo. Integer volutpat eu diam | |
sed iaculis. | |
\chapter{My First Chapter} | |
\label{cha:cha1} | |
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam nec | |
lacus nec dui semper aliquet sit amet in leo. Integer volutpat eu diam | |
sed iaculis. | |
\input{orgChapter1} | |
\chapter{My Second Chapter} | |
\label{cha:cha2} | |
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam nec | |
lacus nec dui semper aliquet sit amet in leo. Integer volutpat eu diam | |
sed iaculis. | |
\input{orgChapter2} | |
%%% Local Variables: | |
%%% TeX-master: "main.tex" | |
%%% End: |
A master LaTeX file for the book
Finally, there is a master file that includes the LaTeX preamble and several \include
commands to import the previous parent files. It is important to note that the parent LaTeX files include three lines at the end.
%%% Local Variables: %%% TeX-master: "main.tex" %%% End:
This code is used by AUCTeX to work with a multifile structure and run commands on the master file.
The master file loads the memoir class, a powerful class extremely useful for designing books. It offers an extensive manual with more than 500 pages containing also examples for the design of a book and of a thesis.
The code blocks are highlighted with the listings package with this configuration:
\lstset{ %keywordstyle=\color{Blue}, commentstyle=\color{gray!90}, % stringstyle=\color{OliveGreen}, basicstyle=\ttfamily\small, columns=fullflexible, breaklines=true, linewidth=\textwidth, backgroundcolor=\color{gray!10}, basewidth={0.5em,0.4em}, % frame=single, literate={á}{{\'a}}1 {ñ}{{\~n}}1 {é}{{\'e}}1 {ó}{{\'o}}1 {º}{{\textordmasculine}}1 }
This is the master file:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
\documentclass[smallroyalvopaper]{memoir} | |
\settypeblocksize{6.5in}{4.5in}{*} | |
\setlrmargins{*}{*}{1} | |
\checkandfixthelayout | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Misc packages | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage[english]{babel} | |
\usepackage[usenames,dvipsnames]{xcolor} | |
\usepackage[T1]{fontenc} | |
\usepackage[utf8]{inputenc} | |
\usepackage[noprefix]{nomencl} | |
\usepackage{url} | |
\usepackage{amssymb} | |
\usepackage{amsmath} | |
\usepackage{graphicx} | |
\usepackage{makeidx} | |
\usepackage{pifont} | |
\usepackage{fourier} | |
\usepackage{siunitx} | |
\usepackage{textcomp} | |
\usepackage{mathpazo} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Bibliography | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage[citestyle=authoryear,bibstyle=authoryear,doi=true,url=true]{biblatex} | |
\let\cite\parencite | |
\addbibresource{myBib.bib} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Memoir config | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage{memhfixc} | |
\usepackage{mempatch} | |
\raggedbottom | |
\sloppybottom | |
\clubpenalty=10000 | |
\widowpenalty=10000 | |
\feetbelowfloat | |
\chapterstyle{ger} | |
\setlength{\afterchapskip}{35pt} | |
\maxtocdepth{section} | |
\setsecnumdepth{subsubsection} | |
\renewcommand{\topfraction}{0.85} | |
\renewcommand{\bottomfraction}{0.5} | |
\renewcommand{\textfraction}{0.15} | |
\renewcommand{\floatpagefraction}{0.7} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Hyperref | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage{hyperref} | |
\hypersetup{ | |
bookmarks=true, % show bookmarks bar? | |
bookmarksnumbered=false, | |
bookmarksopen=false, | |
breaklinks=true, | |
backref=true, | |
pdftoolbar=true, % show Acrobat’s toolbar? | |
pdfmenubar=true, % show Acrobat’s menu? | |
pdffitwindow=false, % window fit to page when opened | |
pdfstartview={FitH}, % fits the width of the page to the window | |
pdftitle={My Title}, % title | |
pdfauthor={My Name}, % author | |
pdfcreator={AucTeX/Emacs}, % creator of the document | |
pdfproducer={LaTeX}, % producer of the document | |
pdfnewwindow=true, % links in new window | |
pdfborder={0 0 0}, | |
colorlinks=true, % false: boxed links; true: colored links | |
linkcolor=black, % color of internal links | |
citecolor=black, % color of links to bibliography | |
filecolor=black, % color of file links | |
urlcolor=Blue % color of external links | |
} | |
\usepackage{breakurl} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Figure and table environment and captions | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\captionnamefont{\scshape} | |
\makeatletter | |
\renewenvironment{figure}[1][]{% | |
\@float{figure}% | |
\precaption{\rule{\linewidth}{0.4pt}\par} | |
\centering | |
}{% | |
\end@float | |
} | |
\makeatother | |
\makeatletter | |
\renewenvironment{table}[1][]{% | |
\@float{table}% | |
\postcaption{\rule{\linewidth}{0.4pt}\par} | |
\centering | |
}{% | |
\end@float | |
} | |
\makeatother | |
\renewcommand{\textfloatsep}{10pt}%Espacio entre el flotante y el texto | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Code printing with listings | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage{listings} | |
\lstset{ | |
%keywordstyle=\color{Blue}, | |
commentstyle=\color{gray!90}, | |
% stringstyle=\color{OliveGreen}, | |
basicstyle=\ttfamily\small, | |
columns=fullflexible, | |
breaklines=true, | |
linewidth=\textwidth, | |
backgroundcolor=\color{gray!10}, | |
basewidth={0.5em,0.4em}, | |
% frame=single, | |
literate={á}{{\'a}}1 {ñ}{{\~n}}1 {é}{{\'e}}1 {ó}{{\'o}}1 {º}{{\textordmasculine}}1 | |
} | |
\renewcommand{\lstlistingname}{Code} | |
\usepackage{fancyvrb} | |
\DefineVerbatimEnvironment{verbatim}{Verbatim}{% | |
fontsize=\small, | |
formatcom = {\color{gray!97}}} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Title, author and date format | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\pretitle{\vfill\begin{flushright}\bfseries\scshape\HUGE\color{BrickRed}} | |
\posttitle{\par\end{flushright}} | |
\preauthor{\begin{flushright}\large\scshape} | |
\postauthor{\par\end{flushright}} | |
\predate{\vfill\begin{flushright}\large\scshape} | |
\postdate{\par\end{flushright}\vfill} | |
\makeindex | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Hyphenation rules for code inside texttt | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\usepackage[htt]{hyphenat} | |
\hyphenation{Spatial-Points} | |
\hyphenation{Spatial-Pixels} | |
\hyphenation{Spatial-Grid} | |
\hyphenation{Spatial-Lines} | |
\hyphenation{Spatial-Polygons} | |
\hyphenation{Spatial-Points-Data-Frame} | |
\hyphenation{Spatial-Lines-Data-Frame} | |
\hyphenation{Spatial-Polygons-Data-Frame} | |
\hyphenation{Raster-Layer} | |
\hyphenation{Raster-Stack} | |
\hyphenation{Raster-Brick} | |
\hyphenation{read-Shape-Poly} | |
\hyphenation{read-Shape-Points} | |
\hyphenation{read-Shape-Lines} | |
\hyphenation{write-Points-Shape} | |
\hyphenation{write-Lines-Shape} | |
\hyphenation{write-Poly-Shape} | |
\hyphenation{union-Spatial-Polygons} | |
\hyphenation{Open-Street-Maps} | |
\hyphenation{Java-Script} | |
\hyphenation{space-time} | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%%End of Preamble | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
%% Body | |
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | |
\begin{document} | |
\begin{titlingpage} | |
\title{The Title} | |
\author{My Name} | |
\date{March 10th, 2014} | |
\maketitle | |
\end{titlingpage} | |
\frontmatter | |
\tableofcontents | |
\clearpage | |
\mainmatter | |
\include{part1} | |
\include{part2} | |
\backmatter | |
\printbibliography | |
\clearpage | |
\printindex | |
\end{document} |
error dijo:
Excelente entrada. Hace falta información con este nivel de calidad para que la gente empiece a ver la potencia de Emacs.
Tan sólo un comentario, ¿por qué no lo escribes en castellano?. Muchas de las personas con las que hablo «se quejan» de que toda la documentación interesante está en inglés (y con razón). Yo creo que lo suyo es que esté en ambas lenguas, y sobretodo si los que escriben tienen la posibilidad de hacerlo en su leguaje natal.
En cualquier caso, muchísimas gracias.
Excellent entry. Is needed information with this level of quality so that people begin to see Emacs power.
Just one comment, why do not you write in Castilian?. Lot of people «complain» that all the interesting documentation is written in English (rightly so). I believe the best is write in both languages, and especially if the writers are writting in his native leguaje.
In any case, thank you.
Oscar Perpiñán Lamigueiro dijo:
Gracias por tu realimentación.
Sobre el tema del idioma, es un problema de limitación de tiempo. Si tuviese más tiempo probablemente realizaría artículos en las dos lenguas, pero no es así. En este contexto pienso en el idioma como una herramienta, y utilizo la más conveniente para que el mensaje de forma efectiva. Y hasta donde yo alcanzo a ver, la comunidad de usuarios de emacs se comunica de forma abrumadora en inglés.
Pingback: Emacs for creative writing | I love Emacs
Mauricio dijo:
Hi Oscar,
Thank you very much for sharing your experience.
However, at the beginning of the post I missed some discussion about «why» using Emacs with orgmode instead of R+ Knitr or R+Sweave. If you could add something it would be great.
Oscar Perpiñán Lamigueiro dijo:
In my opinion, it is a matter of personal preferences. I love working with Emacs, ESS, and org-mode, and I think this combination is extremely powerful. However, I understand that Emacs is not for everyone.
Pingback: Weekly review: Week ending March 21, 2014 - sacha chua :: living an awesome life
Pingback: Writing Books with Org Mode | Wisdom and Wonder
Pingback: Moving to ggplot2 from gnuplot | Mesh Mess
Pingback: Writing Books with Org Mode | Irreal
Pingback: Series temporales, datos espaciales y espacio-temporales con R – datanalytics
Pingback: Displaying time series, spatial, and space-time data with R is available for pre-order ← Patient 2 Earn
Pingback: Writing a book with a little help from Emacs and friends ← Patient 2 Earn
Pingback: Writing a book with a little help from Emacs an...