Parent directories in tar archives

Jan 3, 2014
Tested on: Linux Mint 13 Maya (based on Ubuntu 12.04 LTS)

It is common to see full directory tree or . (dot) in tar archives. Sometimes, it may be unnecessary hassle to remove all parent directories after extracting or to look for the extracted file which merged with other files in extracted directory. In some situations, it may be desirable to create a tar archive with top directory (with informative name). There has been many discussion about the same. Here is quick summary with an example for future reference.

Test setup

We will create a test directory with some files in it with following commands (outputs are also shown): $ mkdir -p /tmp/path/to/test/dir_archive $ touch /tmp/path/to/test/dir_archive/file{1..10} $ ls /tmp/path/to/test/dir_archive/ file1 file10 file2 file3 file4 file5 file6 file7 file8 file9 $ tar --version tar (GNU tar) 1.26 Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later . This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by John Gilmore and Jay Fenlason.

Full parent directory

Following command creates tar archive with full parent directory structure as shown by --list command. $ tar -caf /tmp/test.tar.gz /tmp/path/to/test/dir_archive $ tar --list -af /tmp/test.tar.gz tmp/path/to/test/dir_archive/ tmp/path/to/test/dir_archive/file3 tmp/path/to/test/dir_archive/file10 tmp/path/to/test/dir_archive/file1 tmp/path/to/test/dir_archive/file8 tmp/path/to/test/dir_archive/file4 tmp/path/to/test/dir_archive/file2 tmp/path/to/test/dir_archive/file7 tmp/path/to/test/dir_archive/file5 tmp/path/to/test/dir_archive/file6 tmp/path/to/test/dir_archive/file9

Dot (.) as parent directory

Following command creates tar archive with dot (.) as parent directory. Note that the tar command ends in a dot (.). $ tar --directory /tmp/path/to/test/dir_archive -caf /tmp/test.tar.gz . $ tar --list -af /tmp/test.tar.gz ./ ./file3 ./file10 ./file1 ./file8 ./file4 ./file2 ./file7 ./file5 ./file6 ./file9

Directory name as parent directory

The trick to get dir_archive as parent directory in tar archive is to use --directory to change directory to one level up as shown below. Note location of dir_archive in the command. $ tar --directory /tmp/path/to/test -caf /tmp/test.tar.gz dir_archive $ tar --list -af /tmp/test.tar.gz dir_archive/ dir_archive/file3 dir_archive/file10 dir_archive/file1 dir_archive/file8 dir_archive/file4 dir_archive/file2 dir_archive/file7 dir_archive/file5 dir_archive/file6 dir_archive/file9
Some more notes about tar command:
  • Above commands can be combined with used along with exclude directive to by using --exclude='dir_archive/dir_exclude' after the --directory option.
  • Do not mix absolute and relative path in tar command for pathname and exclude option. --directory is an exception and can be absolute path while others paths can be defined relative to it.
Read more ...

BibTeX hacks

Dec 27, 2013
Sometimes \(\LaTeX\) compilation can give fairly unhelpful error messages like these: ! Missing $ inserted. $ l.190 ... pp. 225--236, 10.1007/978-3-642-31340-0_19 18. [Online]. ? If these error message are in .bbl file, when you are using BibTeX (.bib file) for bibliography then it is possible that these error messages are result of some special characters in your .bib file. I have seen it frequently that some DOI links contain some special characters. Following are some ways to get around these issues so that resulting document is rendered correctly.
  • Escape underscore (_) with a backslash (\). In the above example, the original DOI url was 10.1007/978-3-642-31340-0_19 which was resulting into errors. Escaping the underscore with backslash fixed the issue for me: 10.1007/978-3-642-31340-0\_19 (notice a backslash before _19 near the end).
  • Sometimes DOI urls would contain less-than (<) and greater-than (>) character. These characters do not result into errors during compilation but would be rendered incorrectly. Escaping these characters with backslash generally does not work and often result into errors similar to that shown near the top of the post. An example (fake) DOI url is 2/3<80::AID-HBM2>3.0 which was rendering upside down question mark and exclamation marks (probably because of issues with font encoding). I found that putting < in math mode helps to resolve this issue, i.e. replace with $<$ and $>$ so that new DOI url looks like 2/3$<$80::AID-HBM2$>$3.0.
  • Changing font encoding by adding following lines in \(\LaTeX\) preamble can also help to resolve above issue with < and >, as discussed here. \usepackage[T1]{fontenc} \usepackage{lmodern}
  • \(\LaTeX\) provides means to describe special characters like accents or umlauts using a special notation as described in following table hot-linked from official homepage. In addition, also check out this really great answer on stackexchange about accented letters.

Read more ...

Useful LaTeX tricks

Aug 3, 2013

\(\LaTeX\) word count

It can get difficult to keep track of accurate number of words in a \(\LaTeX\) document. TeXcount is a script for counting words in text and math equations in LaTeX documents. It can also generate very detailed html output with color-coded words types. TeXcount is part of Tex live and can be installed by installing texlive-extra-utils from Ubuntu repository. TeXcount also has a web interface. $ texcount document.tex FILE: document.tex Words in text: 6462 Words in headers: 48 Words in float captions: 274 Number of headers: 19 Number of floats: 4 Number of math inlines: 204 Number of math displayed: 8 texcount -v -html document.tex > word_count.html

\(\LaTeX\) diff

Usual diff programs may not do a great job of diffing two \(\LaTeX\) documents. Following are some of the alternative ways:
  • Use wdiff with colordiff, similar to this stackexchange question: wdiff -n old.tex new.tex | colordiff | less -R
  • Use latexdiff to determine differences and create a new LaTeX document with markup differences between two latex files. latexdiff can be found in Ubuntu repository and detailed description of usage can be found here. An example with beautifully typeset document is given below. latexdiff draft.tex revision.tex > diff.tex
  • Another alternative could be to follow a simple guideline: Always start a new sentence in a new line and do not put any line-breaks in till the sentence ends. This allows traditional diff/merge programs like meld etc to work in a much better way.

\(\LaTeX\) and git

It is useful to use a version control system, like git, with \(\LaTeX\) documents. Here are some useful tips while working with git:
  • Check git + LaTeX workflow for some great tips.
  • Use latexdiff (also see above) in smarter way:
    • latexdiff <(git show checksumHash:file.tex) file.tex > diff.tex
    • Git-latexdiff can also be used for more advanced operations, like documents involving multiple .texfiles.
    • Check stackexchange discussion for more solutions.

\(\LaTeX\) on blogspot.com

MathJax is awesome. It is very easy to install and get \(\LaTeX\) equations working. \[ J_\alpha(x) = \sum\limits_{m=0}^\infty \frac{(-1)^m}{m! \, \Gamma(m + \alpha + 1)}{\left({\frac{x}{2}}\right)}^{2 m + \alpha} \]
Read more ...

Git cheat sheet

Jul 10, 2013

Git overview: Operation and File lifecycle

Image are hotlinked from Pro Git book, written by Scott Chacon.


Some of the commands listed here may use alias and/or configuration defined in ~/.gitconfig (see near the end of the post).

General operations

  • Help with git: git --help # List available git commands git help <command> # More details about a specific command git <command> --help # More details about a specific command
  • Display status of (local) repository: git status git st # Abbreviated using ~/.gitconfig git status -sb # Shorter output git status -uno # Hide details of untracked files
  • Adding files to staging area: git add --dry-run <file1> <file2...fileN> # Dry run. No change in repository git add -n <file1> <file2...fileN> # Abbreviated flag for dry run git add <file1> <file2...fileN> # Add files to staging area git add *.m *.txt # Also supports wildcards
  • Commit changes with message: git commit -m 'message' # Commit ONLY staged files git commit -a -m 'message' # Commit ALL changes, including unstaged files git ci 'message' # Abbreviated using ~/.gitconfig

Compare and inspect

  • Changes in working directory: git diff --color # Colored diff git diff --stat # Show diffstat git diff --ignore-all-space # Ignore all whitespace git diff -w # Abbreviated --ignore-all-space git diff --ignore-space-changes # Ignore change in amount of whitespace git diff -b # Abbreviated --ignore-space-changes git diff -w --color-words # Colored word diff (rather than line diff) ignoring whitespace git diff --cached # Show changes in staged files git diff --staged # Same as --cached
  • Inspect changes:
    (gitk is a tcl/tk based gui; Beginner tutorial) gitk --all # Browse all branches in local repository git log --oneline --graph # Inspect repository with summary of changes git log --oneline --graph --stat # Inspect repository with diffstats git whatchanged --oneline --graph # Similar to git log git lg # Abbreviated using ~/.gitconfig git show <commit> # Shows details of changes in commit
  • Inspect changes beyond file renames:
    (Take care with renames: NEVER commit renames and modifications in the same commit. Check these as well.) git log --oneline --follow <filename> # Summary of changes git log --follow -p <filename> # Detailed changes gitk <filename> # Detailed changes using gitk

Branching

  • Pro Git book has an excellent chapter on Branching and Merging.
  • When typing in the name of a branch in following commands, try using TAB for auto-completion. It works most of the times.
  • git branch # List local branches git branch -a # List all local and remote branches git checkout <branch_name> # Switch to other existing branch git checkout -b <branch_name> # Create a branch and switch to it git branch -d <branch name> # Delete a branch
  • Compare branches: (order of branches matter) git diff <branch1>..<branch2> git diff --stat <branch1>..<branch2> # Generate diffstat git diff --name-status <branch1>..<branch2> # Short diff with only name and status git diff <branch1> <branch2> <filename> # Show differences only for a file git difftool <branch1> <branch2> <filename> # Use (external) diff tool
  • Compare directories (branches/commits): (requires git version >=1.7.10) git difftool --dir-diff <commit1>..<commit2> git difftool --dir-diff <commit1> git difftool --dir-diff <branch1> git difftool --dir-diff <branch1>..<branch2>
  • Remote branches & its management
    A very good and detailed description of remote branch management can be found in Pro Git book. # Pushing to remote git push origin localBranch # Pushes localBranch to origin git push -u origin localBranch # Pushes/sets association of localBranch to origin git push origin localBranch:remoteBranchName # Pushes localBranch to origin/remoteBranchName git push -u origin localBranch:remoteBranchName # Similar effect to -u above # Tracking or defining association. Also see this. git branch -u origin/remoteBranchName # Associates current branch to origin/remoteBranchName git branch -u origin/remoteBranchName localBranch # Associates localBranch to origin/remoteBranchName # Checking out from remote git fetch origin && git checkout remoteBranchName # Checkout origin/remoteBranchName git checkout -b localBranch origin/remoteBranchName # Checkout origin/remoteBranchName to localBranch git pull # Works after checking out or associating. # Deleting remote branch git push origin :remoteBranchName # Deletes origin/remoteBranchName git push origin --delete remoteBranchName # Deletes origin/remoteBranchName # Prune stale branch. See git remote branch deleted but still appears in 'branch -a' git remote prune origin # Removes all stale branches git fetch -p # Fetches from origin and prunes all stale branches # Show details of branch tracking information git remote -v show origin git branch -vv -a

Merge conflicts

Good read about three-way merge in git: Painless Merge... and Three way git... Here is a summary:
  • Use git mergetool with meld. See ~/.gitconfig below. Mergetool would go through the conflicting files and create various new files with different file extensions so that it can use following command with meld (see git source code for more details). meld --output $MERGED $LOCAL $BASE $REMOTE
  • $LOCAL: Contents of the file on the current branch. This is presented in Left-most column in meld.
  • $REMOTE: Contents of the file to be merged. Right-most column in meld.
  • $BASE: Common base of the merge OR half-finished merge where things started to fail. It is shown in middle column in meld. This is column which you would want to edit.
  • $MERGED: File to which meld would write the merge resolution. It is not shown in meld. However, when $BASE is saved after editing, the output is written to $MERGED. Usually, $MERGED does not have any special extension like above files.
  • Mergetool creates *.orig backup files while resolving merges. These are safe to remove once a file has been merged.

Tagging

  • Create/Delete tags (Also see Pro Git book) git tag -a v1.4 -m 'my version 1.4' # Creates annotated tag to current commit git tag -a v1.4 -m 'my version 1.4 <checksum> # Adds tag to an older commit git tag -d <tag_name> # Deletes a tag
  • List tags git tag -l # Lists tags in alphabetical order git tag # Lists tags in alphabetical order git tag -l -n1 # Lists tags along with annotation git tag -l <pattern> # Lists tags in matching a pattern git tag -l v13* # Lists tags starting with v13 git log --tags --simplify-by-decoration --pretty="%d" # Lists tags in chronological order git tl # Abbreviated by ~/.gitconfig git show <tag_name> # Show details of a tags
  • Remote tags git ls-remote --tags origin # List all tags on origin git push origin <tag_name> # Push a tag to origin git push origin --tags # Push all tags to origin git push origin :refs/tags/<tag_name> # Delete a tag on origin

Others:

  • Revert changes (excellent examples in git-reset documentation) git reset # Undo all add. Retains all changes in working tree git reset --hard # Undo add all. Discards all changes in working tree git reset HEAD <file_name> # Undo add one file. Retains all changes in working tree git reset --soft HEAD^ # Undo last commit. Retains all changes in working tree uncommitted (un-stage) all changes including files and folders:


~/.gitconfig

git difftool falls back to git mergetool config variables when the difftool equivalents have not been defined. If a merge resolution program is not specified, git mergetool will use the configuration variable merge.tool. [core] editor = emacs autocrlf = input [merge] tool = meld [alias] ci = commit -a -m st = status -sb rb = rebase lg = log --graph --all --format=format:'%C(auto)%h (%cd)%C(reset)%C(bold red)%d%C(reset) %C(bold green)%s%C(reset) %C(bold white)- %cn%C(reset)' --abbrev-commit --date=short df = diff -w --color-words dft = difftool -y tl = log --tags --simplify-by-decoration --format=format:'%C(auto)%h (%cd) -%C(reset)%C(bold red)%d%C(reset) %C(bold white)- %cn%C(reset)' --date=short --author-date-order [color] ui = true [color "status"] added = yellow changed = red untracked = cyan [push] default = current

Other great Git references:
Read more ...