Cau Dy Geg

… should probably take heed of my own advice above (its Welsh—I'm Welsh!), but its never stopped me yet ...
Posts tagged “pdf”
“

”

Transferring Adobe Digital Editions (ADE) DRM’ed eBooks to iPad

I’m posting this here so that I can find it in the future when I need it again and also so that hopefully Google will shorten the process for anyone that follows along after me down this weary path …

Players

  • Mac OSX 10.6.3

  • Apple iPad (running iPhoneOS 3.2)

  • Apple supplied (free) iBooks app

  • Adobe Digital Editions 1.7.2 for OSX

  • some DRM encumbered eBooks stored in ADE (these are either epub or PDFs)

Tools

  • Calibre (a free eBook management/manipulation tool) [optional]

  • Python v2.6.5 and the pycrypto library v2.1

  • 3x scripts: adeptkeymac.py, adeptepub.pyw and adeptpdf.pyw

Background

Apple have decided on epub as their chosen format for their iBooks application to store/retrieve eBooks, this is to be applauded as epub is an open standard (a little in flux still) and little more than styled HTML, so we expect great things to come. Whilst books generally bought from the Apple BookStore will be DRM encumbered with FairPlay (same system as deployed for their music), addressing that is for another day, the focus here is on Adobe DRM encumbered eBooks. Apples BookStore is still a little light on book content and hence many people have eBooks purchased through other means, Kindle, Sony Readers, Nook, PC/Mac, etc. I’d expect Adobe to be a popular choice …

It’s making some of these available to iPad viewing I’m interested in … some of the more enlightened eBook publishers—-O’Reilly obviously spring to mind here—provide non-DRM encumbered electronic copies of books and/or mechanisms to convert your dead tree purchases into electronic form at reduced rates. The advantages of no DRM on the eBook is flexibility for the owner/reader, they are free to choose their reader/platform of choice and possibly even to pass on the book at a later stage.

Much of the publishing industry is wedded to an Adobe biased delivery workflow (Adobe Content Server), particularly where the production of eBooks is concerned as Adobe has provided them with a suitable container in either PDF or epub for distribution. Most importantly not only have they provided the container, but also appropriate delivery mechanisms for providing a “secure” copy of the publication to the customer.

In Adobe terms, you purchase an eBook and the publisher delivers to you a small .acsm file. This is an Adobe Content Server Message file, which is only applicable to an Adobe desktop tool called Adobe Digital Editions (ADE). ADE is effectively the delivery point for your eBook, your .acsm file is a pointer only to the available download of your eBook from the publisher, in the process of retrieving the actual book from the publisher it is DRMed using a unique key to prevent its viewing or passing to others after you obtain the book.

This in effect ties the book content to you and more annoyingly, to storage and viewing within only the ADE application. As a Mac user, that is particularly annoying as Adobe desktop tools are a pain to install, poorly integrated on OSX and nearly unnecessary for most desktop users with PDF being so well integrated into the OS. Whilst Apple’s own Preview is operationally a much better tool than Reader for most day-to-day tasks..

So, where next …

As an Apple iPad user, I particularly want to take advantage of viewing/reading my books on the iPad and as I’ve paid legitimately for the content, I’m interested in removing any barriers to my enjoyment of my books in that way, namely the DRM on the eBook held in ADE prevents me from just dragging it into iTunes and enjoying it in iBooks.

Well, it did …

What to do …

ADE holds two forms of eBooks; epub and PDF. In both cases the files are stored in open view, in my case ~/Documents/Digital Editions contains each of the DRM encumbered epubs. Just dragging them to iTunes will attempt to transfer them but you’ll find that on sync, iTunes will report that the files are protected and refuse the operation.

So, we need to obtain the key necessary to decrypt the DRM encumbered epub and re-write any epub in a non-DRM format, then possibly tidy up or supplement any metadata in the epub before adding it to iTunes. Thankfully, using a combination of openssl and python, retrieving the key is handled by a short script.

ADE uses a mechanism called ADEPT for its DRM, storing the key in your installation of ADE on the desktop, as all of your purchases will be encrypted with this key, it is both vital and unlikely to change much in the future … so grabbing a copy now and archiving it will potentially be useful until such time as adobe decide to change everybodies keys worldwide (hopefully never!).

Once we have the key available, we need to decrypt the epub or PDF, writing a new version of the document into a non-DRM container. This task is accomplished with a Python script, which utilises the pycrypto library to apply the key decryption to your epub.

FInally, our non-DRM epub is available, but some of the important (but optional) metadata, such as a cover, etc may be left behind in ADE, to re-provision these you can utilise a freely available eBook tool called Calibre, which allows metadata editing and updating.

If you’ve got this far, you are one step from total success, drag the final epub from the workflow into iTunes and sync your device, a couple of seconds later you should be reading your book in peace and quiet on the iPad.

The gory details for OSX

The original process for the task below can be found at http://i-u2665-cabbages.blogspot.com/2009/02/circumventing-adobe-adept-drm-for-epub.html, this is to pull out the OSX specifics and some pointers to modified working scripts for OSX 10.6.3 users.

This works for me, running:

  • OSX 10.6.3

  • Xcode (w. 10.4 support)

  • Python 2.6.5 (additional install)

  • pycrpto library 2.1 (additional install)

  • ADE 1.7.2

  • iTunes 9.1

  • Calibre 0.6.53 (optional)

Steps:

  1. Install the optional 10.4 SDK into current Xcode, this prepares some necessary tools for the later Python/pycrypto toolset.

  2. Download and install the Python 2.6.5 DMG, Apple deliver 2.6.1 and it doesn’t support the pycrypto library we need.

  3. Download pycrypto 2.1, extract it and install it.

    $ python setup.py build

    $ python setup.py install

  4. Install ADE 1.7.2. Registering this with Adobe enables you to share DRMed ebooks across all registered machines (under ADE of course, upto 6 machines), as we’ll be removing DRM you might forgo this. Your key is already available after install, creating and uploading to Adobe just makes it resilient (in the face of you destroying your installation). So choose, whether to register or not, it only affects your future recovery, not the ability to read or process books.

  5. Download your publication (ie. get an .acsm file from publisher), launch it, ADE will then download the epub to its silo.

  6. Obtain the 3x Python scripts you require (Google them … BTW, the versions are minimums to work!):

  7. Run python ineptmac.py, which will generate an adeptkey.der, this is your personal ADEPT keyfile (binary), so keep it safe. Archive a copy now for reference.

  8. With the keyfile in hand, we now run python ineptepub.pyw or its sister script. This will pop up a small window asking for the location of three files, the existing keyfile we created in step. 7, the epub in its current DRM form (typically stored in ~/Documents/Digitial Editions and a filename for the new epub to be created as a result of decryption.

  9. A few seconds later you should have a new non-DRM encumbered epub (or PDF).

  10. [Optionally] Drag the new epub into Calibre and tidy up its metadata, add covers, etc.

  11. Drag the epub file created above into iTunes, which should recognise it and add it to the ebook list, more importantly sync your iPad and start reading!

All the hard work here has been done by others, I just thought the OSX variation should be a little more simply explained given the number of iPad users and the paucity of ebooks in the UK store … until Apple right that on May 28th I’ve happily downloaded a number of current titles in Adobe DRM epub format from The Book Depository, no affiliation, just a very happy client.

More Information