Wednesday, November 28, 2012

Troubleshooting PNG vs JPEG in EPUB

So, I wrote two articles on versioning in iBooks (here and here), but I didn't tell you my main reason for wanting to update the book.

The truth is that in the first version of my Monarch Butterfly book, my favorite page didn't work on the iPad 1. That page has 10 images, each of which measures 1600 x 1100 px. As I was exploring the possibilities of interactivity, it was one of the first pages that I created in the book. And because I had a vague (erroneous) idea that Apple preferred PNG, I decided to experiment with them.

PNG affords some advantages, including allowing transparency, at least in the 24bit version. In the later pages of the book, I used this to advantage for the arrows, so that I didn't have to position them perfectly on top of the images. In addition, PNG doesn't suffer from the licensing issues that JPEG has. On the downside, its compression is not as good, so images are larger.

Here's what the page looks like. It's set up so that if you click the images across the bottom, they are expanded into the center area, so that the reader can view and control a stop-motion movie of the caterpillar emerging from its egg, and then having its first meal. If you click the center image, the animation goes backwards.

Unfortunately, when you open the original version of this page on the iPad 1, it crashes. And like I do so often when I'm troubleshooting—and this is the real lesson of this post—I assumed that the problem was that the iPad 1 was slow and didn't have enough memory, and that I had just pushed it too far by adding too many images. I decided to upload the book as is, and warn people that it simply didn't work well on the iPad 1.

The other day, someone asked me about PNG and the NOOK and I remembered an issue I had come across a year or so ago. I was helping some folks with a book with lots of images. Not particularly big images (it was a flowing book), but lots of them. The NOOK would refuse to let you navigate to the next chapter by turning pages, even though you could choose the chapter from the TOC and it displayed just fine. Very strange. One workaround was to change the images to PNG format.

And even though the problem was the JPEG in that problem, it made me think I should try to switch from PNG to JPEG. And the crazy thing is that it worked. I'm not sure if it's because the images are smaller (233Kb instead of 950Kb), or because of the format itself, but now the page works on iPad 1 beautifully and doesn't crash.

So, the moral of the story is first, don't assume you know what the problem is, and second, let those bits of information percolate in your head and spark new theories that might help—even if they're not directly related, and third, use JPEG images in EPUB for iBooks unless you need PNG's transparency!

P.S. I've just sent an email to everyone who bought the first version of the book so that they can download the new version for free. If you don't receive the email, let me know. If you bought the book through Apple, you'll be able to update the book through the iBookstore: go to Purchased Books, and then click Update next to The Monarch Butterfly Book.

And if you haven't bought the book yet, you can find the latest version on the iBookstore or on my own website.

Tuesday, November 27, 2012

Versioning in iBooks (part 2)

Yesterday, after confirming that the new version of The Monarch Butterfly Book had not yet gone live, I called Apple's iBookstore support line—(877) 206-2092 in the US, see end of post for other countries' numbers—and asked what was going on. The guy was very helpful, told me that everyone had been on vacation between November 16 and November 25th, due to the Thanksgiving holiday (about which he and I both decided we were envious), but that they were back yesterday and should get to it shortly.

He also checked that I had the "What's New" material already included, which I guessed indicated that was a sticking point in many updated versions. (Remember to use the latest version of iTunes Producer to include the What's New information.)

And finally, he said he'd escalate my issue so that it would get solved as soon as possible. I almost, in the interest of science, told him not to escalate it, so that I could see and report on the real process, but sorry, dear reader, in the end, I did not.

About 12 hours later, I got the email from Apple confirming that the new version had gone live.

And this morning, I opened iBooks on my iPad and iPhone to see how those changes would be announced. What I saw was that iBooks adds a number to the Store button to indicate how many updates a reader has available, but doesn't indicate which book is affected:

iBooks updates

iBooks versioning on iPad

When the reader presses the Store link in iBooks on the iPad, they will see Update buttons next to the corresponding books:

iBooks versioning on iPad

On iBooks on the iPhone, going to the Store will display an Updates link:

iBooks versioning on iPhone

And when your reader follows the Update link, they'll see which books can be updated:

iBooks versioning on iPhone What's New?

At that point, and only on the iPhone as far as I could tell, the reader can click the What's New arrow to see what has changed in the new version:

iBooks versioning on iPhone

Once they click the Update button, the new version downloads to their device, and initially will be displayed with the blue "New" banner:

iBooks versioning: New banner

Once the reader has looked at the book, they won't be able to tell which version they have. I couldn't figure out any way to get information about a book's version from within iBooks on the iPad at all, or on any device once the update was downloaded. This is a serious shortfalling for what I believe is a very valuable feature.

The only way for the reader to see the versioning information of an existing book, including what has changed from version to version, is to go to the iBookstore, look up the book, scroll down to the bottom of the Book Description or Details area, and check the What's New section.

iBooks Versioning

The reader can also get a full rundown on the differences from one version to the next by clicking Version History. I think it's rather confusing to use a header like "Version 3.0" when what they mean is that the changes listed are for the EPUB3 version of the book. It is only below that you see the version number that I gave the book: 1.1.

Therefore, you might want to add version info to one of the pages of your book (the copyright page perhaps?) if you want readers to be able to tell which version they have. (I'll do this next time.)

iBooks Versioning info

So, still some wrinkles to be ironed out, but I continue to think that versioning is a brilliant idea. Now I have to update some of my other books so I can see what happens with notes and bookmarks (which aren't allowed in a Fixed Layout book like this one.

There were two major errors that were fixed in this version of The Monarch Butterfly book: some arrows that were in the wrong place on page 24, and something much more important, which I'll discuss in my next post, about JPEGs and PNGs.

You can buy The Monarch Butterfly Book on the iBookstore (and thus be notified as described above about new versions), or directly from me (via Kagi) (in which case I'll notify you personally about new versions to this book as well as any new books via email).

Here are the iBookstore support telephone numbers:

Australia: 1300 307 504 (Note that this is a low tariff number.)
France: 0805 540 117
Germany: 0800 664 5307
Italy: 800 915 902
Netherlands: 0800 0201 578
Spain: 900 812 687
U.K.: 0800 975 0615
U.S.: +1 (877) 206-2092
Toll-free from U.S. and Canada.

Monday, November 26, 2012

Links to Maps from iBooks

A reader asks what happens with the Map links in the Barcelona Beyond Gaudí book now that iOS no longer has the Google Maps app. It's a good question, so I thought I'd answer it here.

In the Barcelona Beyond Gaudí book, I targeted the map request to Google:

<p><a href="#left" id="caption">A spiral decoration on the gates to Sant Pau Hospital. </a><a href=",2.174297+(Spiral%20Decoration%20on%20Gate)" ><img src="images/map.gif" alt="map button" id="mapbutton" /></a></p>

And it looks like this:

Maps in ebooks in iBooks

When your reader clicks the Map link in iOS5, iBooks passes them to the Google Maps app. In iOS6, iBooks passes such links to the Google Maps page, in Safari:

Maps in ebooks in iBooks

And the reader can still click on the Directions or Street View buttons at the bottom. Here is Street View:

Maps in ebooks on iBooks

Then a quick four finger flick, and the reader can jump back to iBooks.

To send the reader to the Apple maps app instead of the Google Maps page, you just have to target the link to the Apple maps server:

<p><a href="#left" id="caption">A spiral decoration on the gates to Sant Pau Hospital. </a><a href=",2.174297" ><img src="images/map.gif" alt="map button" id="mapbutton" /></a></p>

Then when your reader clicks the Map link, iBooks sends them to the Apple Maps app (without asking if they want to leave the iBooks app, one should note).

Note that the Apple Maps app doesn't mark where the location is on the map:

Apple Maps app from iBooks

And there doesn't seem to be a street view, though you can click the page curl in the bottom-right corner to view Satellite view and then zoom in.

Maps app from iBooks

You can get more information about creating links to the Apple Maps app in the iOS Developer Library.

Friday, November 16, 2012

Versioning in iBooks 3 (part 1)

Another promising new feature announced by Tim Cook about iBooks 3 is the ability to make updates to an ebook, and then make those updates available to readers who have already bought the book. There were a few minor errors in my Monarch Butterfly book so I decided I would use it as a guinea pig to test versioning.

The first thing to note is that versioning only exists for EPUB 3 documents. They can be flowing or fixed layout, but they have to be EPUB 3.

So, my first task was to update the Monarch Butterfly fixed layout book to EPUB 3.

Changes to the OPF file

I began with the OPF file, adding version="3.0" to the initial package element and adding the namespaces that Apple recommends.

I updated the title of my book to indicate the version number, though I'm not sure this is necessary (or desired).

I added the required modification date and time:

<meta property="dcterms:modified">2012-11-06T12:00:00Z</meta>

Pay special attention to the time format: CCYY-MM-DDTHH:MM:SSZ. I tried putting just the year, and EpubCheck gave me an error. Note that both the T and the Z are required.

To make sure my fonts are recognized, Apple now requires this line instead of the file:

<meta property="ibooks:specified-fonts">true</meta>

And then comes the part where Apple lets you specify the version of your book. You are allowed two dots and three components (so 1.2.1) and up to four numbers per component (which seems excessive, frankly). And these aren't decimal numbers: trailing 0's count to keep them ordered; 1.10 is later than 1.9.

<meta property="ibooks:version">1.1</meta><!--controls versioning in iBooks3-->

Then, since my book is fixed layout, I had to add the EPUB 3 specific metadata:

<meta property="rendition:layout">pre-paginated</meta>
<meta property="rendition:spread">auto</meta>
<meta property="rendition:orientation">auto</meta>

Because my books' pages include Javascript, I had to add properties="scripted" to each of the pages which links to a Javascript file, even if it didn't use Javascript in that particular page.

<item id="milkweed" href="milkweed.xhtml" media-type="application/xhtml+xml" properties="scripted" />

And then I removed the guide section, which is no longer required by Apple (and was already deprecated in the EPUB 3 spec).

<!--new to 1.1 guide is removed -->

New TOC file

EPUB 3 requires a new XHTML-based TOC file, which is a lot easier to write than the old toc.ncx format, since it's just formed by an ol list with links to the various TOC entries. For a Fixed Layout book, for which you want iBooks to create its graphic TOC (made of the covers of the books), just create a single entry to the first page, or cover in my case:

<nav id="toc" epub:type="toc">
<h1 class="chapter">Table of Contents</h1>
<li><a href="title.xhtml">The Monarch Butterfly Book</a></li>

Then, as a replacement to the guide section, add a landmarks section, which works very similarly:

<nav epub:type="landmarks">
<li><a epub:type="ibooks:reader-start-page" href="title.xhtml">The Monarch Butterfly Book</a></li>
<li><a epub:type="cover" href="title.xhtml">Cover</a></li>
<li><a epub:type="bodymatter" href="milkweed.xhtml">First page</a></li>

Finally, declare the new toc.xhtml file in the OPF file, paying special attention to the properties attribute:

<item id="toc" href="toc.xhtml" media-type="application/xhtml+xml" properties="nav"/><!--new in 1.1-->

That's it for the OPF file.

Changes to the XHTML files

In each of the XHTML files, I had to change the meta tag where I declared the character set. It was:

<meta content="text/html; charset=UTF-8" />

and it should be:

<meta charset="UTF-8"/>

I also had a few documents that somehow still had extraneous DOCTYPE elements. I got rid of those and made sure that each XHTML document started like this:

<?xml version="1.0" ?>
<html xmlns:epub="" xmlns="">

There were also a few places I had erroneously used character entities, things like &copy; to show the © symbol. Now that I had the proper character set declaration, that was no longer necessary (or allowed). So I simply removed the character entities and replaced them with the characters themselves, typed in directly.

<p>Copyright ©2012 by Elizabeth Castro...

As far as content, I also fixed the error on page 24, which had the blinking arrows in the wrong place:

Wrong arrow placement

And put them where they go:

Correct placement of arrows

I also changed the Javascript so that the arrows would blink more slowly. People seem to have had difficulty finding the captions. It's actually a tricky problem. I want the photos to be displayed without distractions, but I want to offer clues. I haven't quite found the perfect solution yet.


Once of all of the changes were made, I used EpubCheck to make sure I had done it all correctly. I used the latest version: EpubCheck3RC.

Uploading the new file to the iBookstore

Finally, I uploaded it to Apple's iBookstore with iTunes Producer. When you make changes to a book that already exists, you only have to change the parts in iBooks Producer that are different, and it will only update those pieces of your package. So, I updated the description and then on the Assets page, I clicked the Choose button under Publication, and selected the new updated EPUB file. This screenshot is actually after I've done that. iTunes Producer doesn't give a lot of feedback.

Upload new file in iTunes Producer

And then I got the error I was suspecting:

Versioning error

I was expecting it because Apple says that you have to say what changed in a "Version_whats_new" XML file. I hadn't been able to find where that was.

It turns out I had downloaded the latest version of iTunes Producer, but I hadn't installed it. Once I did that, I was able to add the What's new information on the Book page:

iTunes Producer: What's New

Then I delivered the package again (no need to upload the book files again), and it went through without a hitch.

What I am most interested to see now is how it notifies me (and all of you who have purchased the book through the iBookstore) about the new changes. The new description went live almost immediately, but when I go to iTunes Connect, it tells me that the EPUB 3 version is under review.

iTunes Connect-3

And indeed, I don't see that little update icon that Tim Cook promised. Though I'm left with questions: When does iBooks check? Does versioning only work in iBooks 3? Do I have to open and close the app to force a check?

Update versioning, not yet

When I go to the Purchased section of the iBookstore, I also see that it shows that the book was already downloaded and I have no opportunity to update it. I assume that when the changes go live, this button will change.

Already downloaded

If you haven't bought the book yet, you'll be able to tell if you have the new version by checking page 24 and clicking the info arrow to see where those arrows show up.

And I'll keep you posted about how long it takes to go live. It seems like a really important feature to me. I would have liked to add a note or bookmark to see if those are maintained, but fixed layout format doesn't allow them. I'll have to update one of my other books...

Saturday, November 10, 2012

Zen* and the Art of the Modular Ebook - #BiB12

I spoke at the amazing Books and Browsers 2012 conference, organized by Peter Brantley of the Internet Archive, at the end of last month in San Francisco. They've just made the video of my talk available so I wanted to share it.

In the video, I share the experiences with print publishing, Barcelona, and HTML that led me to where I am today—a bestselling computer book writer, turned publisher and Catalanist!

I should note that my blog is misspelled in the titles. If you're here, you know it's Pigs, Gourds, and Wikis.

And I'm particularly embarrassed that I misspoke about Matthew Tree. He is not "London-based" but rather he is originally from London, but has lived in Barcelona for the past 25 years or so. His book, Barcelona, Catalonia: A View from the Inside, has been described as “A must-read to understand Barcelona and Catalonia”.

And La Biblia del Macintosh, Tercera Edición, which was one of the first ever books about the Macintosh to be translated into Spanish actually had 1000 pages, not 500. It sold for 5000 pesetas, which was about $50, a monstrous sum at the time. I think we did an initial print run of about 3000 copies, which we financed by localizing the first ever translation of Adobe Photoshop into Spanish (version 3). We eventually sold all 3000 copies, taking advantage of the social marketing tools of the time: email, newsletters, and expensive advertisements in the Spanish edition of MacUser. I can only imagine where we could have gotten to if we had had Twitter.

They say you get less nervous the more that you speak in public. I'm still waiting!

Friday, November 2, 2012

OverDrive is terrible and discriminatory

So I've got a sick kid, and he wanted to listen to Ender's Game, by Orson Scott Card. I pop over to the Boston Public Library which has an extensive collection of audio books, and ho! I'm in luck. They have it and it's available. I check it out, click the Download button, it downloads, and then I get this:

OverDrive Media Console

Yeah, I had seen that the file was a WMA file but since it said iPod as well, I thought I might be able to do it, oh my gosh, FROM MY MAC. But no. Turns out Overdrive discriminates against you if you have a Mac. I love libraries, but BPL: this is terrible.

Frustrated, but as yet undaunted, I fire up Parallels, so I can run Windows, download a brand new version of the Overdrive Media Console for Windows, which presumably supports said file type. Sign into BPL again, and download the book again. I struggle for a bit because it wants to open it up in the Mac version of the console, even though we have already seen that that won't work.

Finally, I get it to open up in the Windows version of the Overdrive Media Console, and I get this:

OverDrive Media Console

Yeah, I couldn't read it either. All I can think is, boy, it's a good thing I have a kind of wide screen monitor. Can you believe that?

With some trepidation, I click OK. I go back to the Overdrive Media Console on Windows and, as instructed, click Tools > Windows > Windows Media Player Security Upgrade. I get another big, wide error:

OverDrive Media Console

I say OK again, even though it's practically impossible to read.

Yet another window pops up and tells me that more DRM security files are going to be sent to who knows where to keep track of this audio file. For pity's sake.

Security Upgrade Required

I say OK again. In for an inch, in for a mile. And get this for my troubles:

Windows Media Player

Now, if you believe there's a server somewhere handling this that has ever been available, I've got a bridge to sell you. It's ridiculous. Ridiculous that Overdrive and the BPL treat me like a thief, ridiculous that I can't borrow books from the library as a Mac user, ridiculous that Overdrive makes me update a program I downloaded today, makes me jump through various incomprehensible hoops to do so, wastes an hour of my time, and at the very end says, "Oh, too bad, we don't care enough to make the process actually work."

Overdrive, you do a terrible disservice to libraries and book lovers. BPL, I love you, and I have continually advocated paying more taxes that should go to support what you do, but Overdrive does not serve me and it makes you look terrible.

And iBooks needs iOS6 for sharing, too

One of the new features added to iBooks 3 that I wanted to try was sharing bits of a book via Twitter. But when I tried it, it didn't work. I thought maybe it only worked with purchased books, or DRM'd books, or some other kind of book I didn't have. Nope, the problem once again, as with MathML, hinted at by Baldur Bjarnason, was that I was still resisting upgrading to iOS6.

Now that I've got iBooks 3 and iOS6, sharing bits of books via Twitter (and Facebook, Mail, or Message) works swimmingly. Start by selecting the text you want to share.

Once the text is selected you can either highlight it or share it without highlighting (by clicking Share at the far right). I want to highlight first, so I click Highlight.

Sharing with iBooks3 - Select

Once the text is highlighted, iBooks offers me additional options. I can change the color of the highlight, remove the highlight, add a note, or share—with the Away arrow—which I now click.

Highlighted - Share

Once you click the Away button, iBooks offers you a choice of sharing methods. I'm going to use Twitter.

Sharing with iBooks - Twitter

Then iBooks will show you the selection (or a piece of it if it doesn't fit within Twitter's 140 character limit), along with the iTunes link to the book, if it's a purchased book. You can share from sideloaded (e.g., non-purchased books) but iBooks won't be able to link to them in the iBookstore.

Sharing - selection

You can add your own comments (as long as they fit). And, of course, you could edit or remove the shared bit or the link as well.

Sharing - comments

Finally, click the Send button.

If you've highlighted before sharing, that's what you'll be left with.

Sharing - highlighted

The tweet becomes part of your regular Twitter stream, as if you had used your regular Twitter client. This was just an example, and I didn't expect a response, but Twitter is like that, and someone was listening:

Tweeting iBooks

Indeed, the more I think about it, the more I like the idea of tweeting interesting parts of books that I'm reading. I imagine I'd do it less with fiction than with non-fiction, but I'll let you know.

Twitter is limited to 140 characters, but I was curious how much iBooks would let me email. I made the text as small as I could, and then selected two full pages of text, and emailed it to myself.

Apple automatically clips the selection to 199 words, and then adds the cover, author, title, publisher, publication date, and copyright notice, along with a link to the book in the iBookstore.

Sharing with iBooks 3 via email

Also notice that the excerpt is centered, and images and all formatting instructions (including bulleted lists) are stripped. For example, if I select a section of my Read Aloud EPUB book as shown here:

Sharing images and formatting

The resulting email looks pretty dreadful:

Sharing images and formatting (or not)

I also tried copying text and pasting it into the Notes app, but that excerpt, too, was clipped to 199 words, and the formatting and images were removed.

I was able to copy successive paragraphs out of books and thus cumulatively copy out more than 199 words. Given that they feel compelled to limit copying, this at least is a sort of reasonable compromise: Keep readers from simply copying the entire book out, but let them share as much as they want in bits and pieces. Note that iBooks limits how much can be copied whether the book is DRM protected or not.

Finally, I also noticed that no highlighting or sharing is allowed in fixed layout books. I don't know why not. Seems like a recipe book, which might well be in fixed layout, would be just the place you want to add notes and perhaps comment on what you're making on Twitter.

iBooks3 sharing fxl

Thursday, November 1, 2012

iBooks3 needs iOS6 to support MathML

One of the promises that Tim Cook made in his quick demonstration of iBooks3 last Tuesday was support for MathML. The example he showed was created with iBooks Author, and since I'm still not a convert, I tried a Math textbook not created with iBA as soon as I could after downloading iBooks3.

And was disappointed.

Here's a screenshot that I took with iBooks3 on iOS5:

MathML in iBooks 3 on iOS5

Notice that the alignment is off, and several characters cannot be properly displayed and so are shown with empty boxes instead. Icks.

So I wondered if there was something magic (or tricky) about the way iBooks Author creates the MathML. But everything I heard said that wasn't the case.

I figured out the answer yesterday. iBooks 3 needs iOS6 to support MathML.

Here is the same section of the book (though now slightly repaginated), after upgrading to iOS6 (but with the exact same version of iBooks3):

MathML in iBooks3 in iOS6

MathML in iBooks3 with iOS6

Of course, the beauty of MathML is that the equations are just text, and thus can be made bigger or smaller to suit the reader. When I bump up the text size, the equation maintains its beautiful formatting:

MathML in iBooks3 in iOS6 (big!)

MathML, like Asian language support, is one of the key features that make EPUB3 such an important standard. Without MathML, there are no math or science textbooks.

I hear there are other new features in iBooks3 dependent on iOS6. I will continue to highlight them here. If you subscribe to this blog, I'll send my new updates right to your mailbox.

More of my books