Dropbox Bug?

UPDATE: This is an expected behavior of dropbox. See relevant discussion at Dropbox Forums.

It seems that there is a dropbox bug which may affect the recovery of deleted files and folders in this specific situation. I will try to document and elaborate my observations in this post. In short, dropbox seems to be not able to recover a deleted folder if a new file with exact same name is replaced at the exact same location as the deleted folder. I am a free user of dropbox and I am not sure whether this is one of the free features. I found this particular problem 7-8 months back & I could recover the files from my personal backup, which is separate and independent from dropbox.

Here are the steps to reproduce the results. (I did backup all the data, so nothing was lost in reality. This is just a demonstration.) Here, I am going to delete a folder named 'Image Fun' from my dropbox which has some more files and folders in it. The goal is to check, if I can recover those files and folders inside the folder Image Fun.

1. Make sure Dropbox is running. (I am using Linux Mint 13)
2. Delete the folder Image Fun, with some files and folders.
3. Create a file with name Image Fun (no extensions) somewhere outside the Dropbox folder and then copy the file to the location of original Image Fun folder.
4. Log on to dropbox.com website and hit "Show deleted files". I can not see any folder which has been deleted & has name Image Fun. Though, I can see the file I created with the exact same name (among normal files, not deleted files).
5. Navigate to "Events" link on the left on dropbox website. This is the part where it gets interesting. I could see the notification which says that I deleted Image Fun and other 45 files in that folder. See the screenshot below. Now when I click on Image Fun, I am taken to home page where it highlighted the new Image Fun file.


6. Clicking on "output" link in events page (output was a folder inside the delete folder) showed a message in red saying "The folder /Image Fun/output is deleted" (screenshot below).


7. When I clicked on "45 more files" in the events page, it showed me a page where all the files were listed, but I could not click on any of them in order to recover. 


8. But, when I clicked on link "a1.jpg" in events page (where it said "You deleted a1.jpg and 45 more files"), I was taken to a page where I can actually see the older version of that particular file - a1.jpg. Clicking on version 0 shows the correct original file. 

9. You must have noticed the "Restore" button on a1.jpg link (above screenshot). When I clicked on it, something unexpected happened. The file which I created, with name Image Fun, was actually removed with the folder Image Fun and a1.jpg was restored in the folder. Now I could navigate to the folder and even see deleted files. This could be one way to recover the file. But I would count it as very unreliable method. It relies on finding and clicking on one particular file's deleted link and then recovering it. That particular link may actually get lost if many other files in other locations were also deleted. Events page after the recovery looks like this:

Hopefully, this will help to fix or address the issue I mentioned. It is probably an expected behavior. See relevant discussion at Dropbox Forums.

Read more ...

gs (GhostScript) cheat sheet

Cheat-sheet for versatile interpreter GhostScript. It can be used to tweak, convert, produce high quality Postscript and PDF files.

Convert postscript (ps) to pdf:

gs -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=fileout.pdf filein.ps

Merge ps and/or pdf:

gs -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=fileout.pdf filein1.ps filein2.pdf filein3.pdf

Extract page(s) from a ps or a pdf document:

gs -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dFirstPage=3 -dLastPage=3 -sOutputFile=fileout.pdf filein.ps

Embed fonts in a pdf:

gs -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=fileout.pdf -dPDFSETTINGS=/prepress -dEmbedAllFonts=true -dSubsetFonts=true -dCompatibilityLevel=1.6 filein.pdf

Convert images to pdf:

Check this post. Or else use ImageMagic's convert.

PDF optimization options

-dPDFSETTINGS=/screen (screen-view-only quality, 72 dpi images) -dPDFSETTINGS=/ebook (low quality, 150 dpi images) -dPDFSETTINGS=/printer (high quality, 300 dpi images) -dPDFSETTINGS=/prepress (high quality, color preserving, 300 dpi imgs) -dPDFSETTINGS=/default (almost identical to /screen)

Paper size options

-sPAPERSIZE=letter -sPAPERSIZE=a4 -dDEVICEWIDTHPOINTS=w -dDEVICEHEIGHTPOINTS=h (point=1/72 of an inch) -dFIXEDMEDIA (force paper size over the PostScript defined size) -gWIDTHxHEIGHT (page size in pixels)

Output devices:

-sDEVICE=pdfwrite -sDEVICE=ps2write -sDEVICE=png16m (24-bit RGB color) -sDEVICE=pnggray (grayscale) -sDEVICE=pngmono (black-and-white) -sDEVICE=pngalpha (32-bit RGBA color) -sDEVICE=jpeg (color JPEG) -sDEVICE=jpeggray (grayscale JPEG) -sDEVICE=epswrite (encapsulated postscript) -sDEVICE=txtwrite (text output, UTF-8) Check -sOutputFile below, for separate images for each page of a multi-page document. Tiff, PNM and many more formats are supported. Check details in Ghostscript output devices.

Help & list of available devices:

gs -h

Other options

-dNOPAUSE (no pause after page) -dBATCH (exit after last file) -sOutputFile=ABC-%03d.pgm (produces 'ABC-001.pgm'..'ABC-010.pgm'..) -dEmbedAllFonts=true -dSubsetFonts=true (Embeds only the characters used in document) -dCompatibilityLevel=1.4 (Adobe's PDF specifications, >=1.4 for font embedding, =1.6 for OpenType font embedding) -dCompressPages=true (compress page content) -dFirstPage=pagenumber -dLastPage=pagenumber -dAutoRotatePages=/PageByPage (or /All or /None) -rXRESxYRES (XRES & YRES in pixels/inch) -rRES (same XRES & YRES, affects images and fonts converted to bitmaps) -sPDFPassword=password More details & defaults on ps2pdf documentation.
Adobe® PDF Creation Settings - Used by GhostScript.
GhostScript Documentation.

Read more ...

Cheat Sheets

Cheat sheets in this blog are not static posts and they keep getting updates from time to time. Here is the list of cheat sheet (from this blog) for different applications:

• Useful \(\LaTeX\) tricks
• BibTex hacks
• Git - distributed version control system
• GhostScript - an interpreter for the PostScript language and PDF
• Infrequent Linux commands
• Emacs - A highly customizable and advanced text editor
• CVS - version control (Not updated anymore)

Some other useful cheat sheet resources: 

• http://www.cheat-sheets.org/
• http://www.quicklycode.com/
• http://devcheatsheet.com/
• http://www.sosmath.com/ - A somewhat detailed & categorized math cheat sheet
• http://tutorial.math.lamar.edu/cheat_table.aspx - Another math cheat sheet

Read more ...

Where did my disk space go?

Have you ever noticed missing disk space on newly bought hard disks? Here are some possible reasons:

1. Unit confusion: The most common confusion in between GB/GiB or MB/MiB.
2. Reserved space in linux partitions.
3. Filesystem overheads.

Unit Confusion

Many people have questions like: Why my 1 TB hard disk shows only 930 GB? This is purely a confusion of units and bases used when words like 'kilo', 'mega' etc. are used. Following table, from AskUbuntu, explains the situation. Prefix Bytes Prefix Bytes 1 Byte = (2^10)^0 = 1 1 Byte = (10^3)^0 = 1 1 Kibibyte(KiB) = (2^10)^1 = 1024 1 Kilobyte(KB) = (10^3)^1 = 1000 1 Mebibyte(MiB) = (2^10)^2 = 1048576 1 Megabyte(MB) = (10^3)^2 = 1000000 1 Gibibyte(GiB) = (2^10)^3 = 1073741824 1 Gigabyte(GB) = (10^3)^3 = 1000000000 1 Tebibyte(TiB) = (2^10)^4 = 1099511627776 1 Terabyte(TB) = (10^3)^4 = 1000000000000 Many softwares and operating systems often calculate size in GiB and put GB when they are displaying it. This becomes even worse because most of the harddisks are labelled with actual GBs and hence when numbers are reported in GiB, it creates the confusion. Online calculators like this can help to remove the confusion. NIST states very interesting historical reason of this confusing prefix:

Once upon a time, computer professionals noticed that 2¹⁰ was very nearly equal to 1000 and started using the SI prefix "kilo" to mean 1024. That worked well enough for a decade or two because everybody who talked kilobytes knew that the term implied 1024 bytes. But, almost overnight a much more numerous "everybody" bought computers, and the trade computer professionals needed to talk to physicists and engineers and even to ordinary people, most of whom know that a kilometer is 1000 meters and a kilogram is 1000 grams.
Then data storage for gigabytes, and even terabytes, became practical, and the storage devices were not constructed on binary trees, which meant that, for many practical purposes, binary arithmetic was less convenient than decimal arithmetic. The result is that today "everybody" does not "know" what a megabyte is. When discussing computer memory, most manufacturers use megabyte to mean 2²⁰ = 1 048 576 bytes, but the manufacturers of computer storage devices usually use the term to mean 1 000 000 bytes. Some designers of local area networks have used megabit per second to mean 1 048 576 bit/s, but all telecommunications engineers use it to mean 10⁶ bit/s. And if two definitions of the megabyte are not enough, a third megabyte of 1 024 000 bytes is the megabyte used to format the familiar 90 mm (3 1/2 inch), "1.44 MB" diskette. The confusion is real, as is the potential for incompatibility in standards and in implemented systems.

Reserved space

While formatting Linux partitions like ext2/ext3/ext4, 5% of the drive's total space is reserved for the super-user (root) so that the operating system can still write to the disk even if it is full. This space is wasted if the drive is being used solely for data storage especially on large partitions. The details of the reserved space can be seen by running following command (replace sda1 by your drive name)sudo tune2fs -l /dev/sda1 The reserved space can be decreased to 1% of the drive's total space by running following command. To completely remove this reserved space replace 1 by 0. sudo tune2fs -m 1 /dev/sda1 As explained in Ubuntu Documentation, using this command does not change any existing data on the drive. You can use it on a drive which already contains data. This should be left to 5% on drives containing filesystem /, /var, /tmp in order to avoid any problems.

Filesystem overheads

Journaling file systems like ext2/ext3/ext4/NTFS used file tables for managing the filesystem. These tables take up lots of space (2-3% of drive size). Different file system allocates space to these tables differently. For example ext4 allocates all its space during the format itself, which shows up as used disk space. Where as NTFS allocates it when more files are written to disk.

The exact details are more complicated than this handy wavy explanation. Many forum posts has discussed this issue.

Read more ...