Creating Interactive Widgets for iBooks Author and the iPad – Part 1

February 16, 2012 in Data Abstract, iOS, JavaScript, Xcode

When Apple unveiled iBooks2 with support for advanced textbooks on the iPad, they also released iBooks Author to make the creation of such advanced textbooks easier. One of the features of iBooks Author is the ability to embed interactive HTML5 & JavaScript widgets created with Dashcode. Unfortunately, there isn’t a lot of details on exactly how to do that.

Since we just released our RemObjects SDK and Data Abstract for JavaScript, I was naturally curious if they could be used to build interactive widgets for iBooks2. This would be a really powerful feature – the ability to have live updated data displayed in an iBook. Since there wasn’t a whole lot of details I figured a lot of it out of the hardway. Since I started, a few details have started to emerge, but no complete references.

This is the first of four articles on creating interactive widgets for iBooks Author and the iPad’s iBooks2.

  1. Using DashCode to create an interactive widget (This article)
  2. Creating an interactive widget without Dashcode
  3. Creating a RemObjects SDK client widget
  4. Creating a Data Abstract client widget

First of all, what is Dashcode? Apple originally provided Dashcode as a tool to create Dashboard widgets for the Mac. Since these widgets are powered by HTML5 and JavaScript, it makes sense that Dashcode also supports creating Safari web applications. With the need for HTML5- and JavaScript-powered interactive widgets in iBooks Author, Apple just reused the existing Dashboard widgets.

Dashcode - New Widgets

Dashcode previously installed with Xcode, but with Xcode’s move to an App in the Mac App Store it is now a separate download and you will need an Apple Developer acount. Once you have it installed, create a Dashboard – Custom Template. This provides you with a simple canvas ready for you to work with. You will notice it has the text “Hello World” in the middle, which is as good a place as any to start. You can click the Run button in Dashcode to see your widget in action. Use the stop button when you are done.

Custom Template - Hello, World!

A quick tour of the Dashcode IDE and what is in your project with a look at how this relates to embedding your widget in an iBook:

Dashcode - New Project

Beyond the toolbar, the main IDE window is divided into four views. From the upper left and moving clockwise:

  1. Main widget navigation – This gives you access to the parts of your widget and set attributes, etc. Selecting something in this area changes what you view to the right.
  2. Widget canvas – From here you can select, add, remove and resize parts of your widget.
  3. Source code – This displays the source code of the file selected in the lower left.
  4. Files list – This is a list of the files in your widget. Selecting one of these changes what you view to the left. Double clicking a file opens it in a new window.

All of these areas, except the first, can be changed from the View icon in the toolbar. If you have a different view in one of these areas, you can change it back through the View icon.

Main Widget Navigation

Each of the items in this view gives you access to a different part of your widget.

Widget Parts

The first node in the list is what gives you access to the widget canvas. It has sub-items indicating all the parts of the widget. When you save your widget to disk, the name you save it as will replace untitled for this first node. As you add new items to your widget from the library, they will show up here as well. There are two parts to your widget: front and back. Typically on a Dashboard widget there is an info button in the lower right that flips the widget over to give you access to settings and about information. You will notice a gold square next to the front since that is what you are currently viewing.

The custom widget template gives you this “info” part that you see there now. The problem with this “info” part is that it only appears when the mouse is over the widget. On the iPad there is no concept of a mouse, so the “info” part never appears. Even if you replace it with an always visible part that triggers the same event, the iPad doesn’t seem to support the flip-over animation. This means the back is never displayed, so don’t put anything important there. So you can remove the “info” part, any JavaScript relevant to it, and everything on the back of your widget, if you like.

Widget Attributes

The second node gives you access to the widget attributes.

Dashcode - Widget Attributes

The widget identifier doesn’t seem to play any significant role for widgets in iBooks. The same is true of the version number.

If your widget is going to communicate with an external server over the Internet, you need to enable the “Allow Network Access” under Network / Disk Access. I’m not sure what the “Allow External File Access” would give you access to on the iPad, likewise for the Extensions and Plugins – these are a Mac OS X-specific feature and I don’t believe there is an equivalent on the iPad.

The Localization section allows you to localize your widget for other languages. When you are specifying text values in the inspector, you specify both the key and the value for your default language. In the future you can provide alternative sets of values for other languages here.

Default Image

When your widget isn’t activated, the Default image is what gets displayed. For example when the widget is displayed on the page and before the user touches it to activate it. Inside the widget package there is an image called default.png. Dashcode automatically updates the default image with all the parts you add to your widget, except for text. This can look kind of odd if your widget includes a lot of text. If you want it to include the text too, you need to select that option in the inspector for your text parts. There may be other parts that are not included by default, and they should be able to be included the same way.

Dashcode - Inspector - Attributes

You also have the option to stop the automatic synchronization or start it again. Or you can import a default image from an external source, or open the default image in an external editor.

One thing to keep in mind: If you don’t have the default image automatically generated by Dashcode, you will want to make sure it is the same size as your widget, or at least the same aspect ratio. Otherwise it will appear incorrectly while in the inactive view.

Widget Icon

The widget icon is used when the widget is displayed in the “manage widgets” screen from the Dashboard. It is never used when the widget is embedded into an iBook.

Run & Share

This is another really important screen for iPad compatibility with your widget.

Dashcode - Run & Share

The “Minimum Mac OS X version for this widget” defaults to 10.4.3, but it needs to be changed to “10.4.0 (backwards-compatible)”. If you don’t do that, all of the AppleClasses that Dashcode uses to create the cool looking parts for your widget will be missing on the iPad. When you select the backwards-compatible version, these libraries are included in your widget, which makes them available on the iPad. Remember, the iPad is running iOS and not Mac OS X, so we cannot assume the latest Mac OS X widget libraries are available.

The Deployed Widget Name and Compression are optional. The Compression is probably a good idea, especially since we are including the AppleClasses libraries for backwards-compatibility. When you are all done designing your widget, you will want to use the “Save to Disk…” button here to save it somewhere you can load it into iBooks Author.

The Files

I already mentioned the default.png and icon.png, the former being required for an iPad widget. Some of the other files worth pointing out include:

  • Info.plist – This is the property list that contains all the attributes and details about your widget. iBooks Author uses this to know how to embed your widget. We will take a closer look at this one in a future article.
  • main.css – This is the cascading style sheet maintained by Dashcode with all the styles necessary for your widget. You can edit it directly if you want to tweak any of the styles beyond what Dashcode provides.
  • main.html – This is the HTML file that contains all the parts of the widget we are working on. Again, feel free to edit it if you need something special, but be careful with the Dashcode parts that are included, they are not standard HTML (remember the AppleClasses that Dashcode uses?).
  • main.js – This contains all the JavaScript to provide the interactivity for your widget. You will be adding JavaScript in here when you create event handlers.
  • parts folder – Any parts you include from the library get included in here automatically.
  • images folder – A good place to keep any other images you add. By default, it contains a “Designed with Dashcode” image for display on the back of the widget, but we don’t need that anymore since we cannot flip to the back.

You can include additional files in your widget if you make use of additional JavaScript libraries, for example. Just drop them in the file area. Or you can create a new file from the File menu.

Creating our Widget

Let’s go ahead and create a widget that does something. When you are ready to add parts to your widget you need to bring the Library up from the toolbar.

Dashcode - Library

The Library contains a large selection of parts to choose from. I’m going to add a Text Field and a Button to my widget. Then I am going to select the button and add a behavior. This is done via the inspector, which you can bring up from the toolbar.

Dashcode - Inspector - Behaviors

This provides a list of Events you can add a Handler for. Notice that on the iPad you cannot deal with the ondblclick or most of the mouse events. So be sure you test your events on the iPad before you commit to using them in your widget design.

I’m going to add a handler for onclick. The easiest way is to double click on the handler side, provide the name of the handler you want to create and then press enter. Alternatively, you can select from an existing handler if one exists by using the arrows to the right.

This will put us in the code window down at the bottom so we can start adding some code to the handler. Assuming you still have the helloText part and used the default part names like I did, your code for the handler should look like the following.

function buttonClick(event)
{
    helloText.innerText = textField.value;
}

I would encourage you to type it instead of copying and pasting since that gives you an opportunity to see Dashcode’s code completion, which is a nice feature for a dynamic language.

Now we can run our widget to see if it works as expected. Dashcode actually provides functionality to include breakpoints as well as stop automatically on errors. Again, nice features for a dynamic language. Assuming everything worked, it should look similar to this:

Widget - Running

One note about debugging: Sometimes it is easy to include an alert statement in your JavaScript as part of debugging. These alert dialogs will work just fine when running on the desktop, but they cause problems on the iPad. When an alert is displayed on the iPad, it appears just fine, but I was unable to dismiss them on my iPad. So make sure you remove any alerts before deploying to the iPad.

Adding to iBooks Author

When your widget has the functionality you want, you are ready to add it to iBooks Author. First remember to go to Run & Share and save it to disk somewhere convenient. Once you have saved it to disk, you will see a black square icon representing your widget. This is actually a package, which is a directory containing multiple files. If you double-click it, you will be prompted to install it in the Dashboard – if you do that, it will get deleted from whereever you saved it. This can be really annoying, so make a backup, especially if you have tweaked the package contents.

Make a Space in iBooks Author

In iBooks Author, you need to add an HTML Widget to your iBooks layout for where you want your interactive widget to appear. Just click Widgets, then select HTML from the drop down, then position it where you want it. You can resize it as well.

Embedding the Widget

There are two ways to embed your widget. You can use the Inspector (on the toolbar). Just go to the last tab for Widget and select the Interaction view. Now you can use the Choose button to locate your widget.

iBooks Author - Inspector - Widget

The other way is to actually drag your widget directly from finder into iBooks Author. You need to make sure you drag it into the white square that says HTML inside the widget area. You will see a blue box around that area when you are in the right spot. Once you have done that, you can actually resize and move your widget inside the box it is in. What you see in iBooks Author is the default image from your widget. So if some text parts are missing, you know you need to go tell Dashcode to include them in your default image.

Widget in iBooks

When you first click the interactive section that contains your widget you will see an Edit HTML button. Interestingly, clicking this button doesn’t let you edit the HTML, but instead lets you run your interactive widget inside iBooks Author.

Deploying

Now it is important to preview your widget on the iPad. To do this you need your iPad attached via a USB cable and the iBooks app running on the iPad. When you are ready, click the Preview button on the toolbar. It takes a few seconds, but the iBook will appear in iBooks and open. From here you can find your widget. You will see it inactive first, which shows the default.png.

Widget on iPad

Once you touch it though, it will switch to a full screen view where you can interact with the widget. You may at this point notice that it doesn’t look exactly the same as it did in Dashcode or on the desktop.

widget interactive

It seems to move the text around a bit when it runs on the iPad. This only seems to happen when working with the Dashcode widget parts though. Straight HTML (as we will see in Part 2) seems to work fine.

If your widget does take keyboard input (as ours does), keep in mind that the widget always remain centered, even when the keyboard appears. This can result in part of the widget being obscured, even if it doesn’t take up the full screen.

Conclusion

Dashcode provides a really powerful IDE for creating and testing widgets that can then be included in an iBook on the iPad. The combined power of HTML5 and JavaScript can result in some really powerful functionality, all embedded in an iBook. Keep the following in mind and you are ready to include interactive widgets in iBooks for iPad.

Things to remember:

  1. Turn backwards compatibility on, or a lot of parts disappear.
  2. The back of the widget never gets displayed.
  3. Text doesn’t show up in the default image unless you change the option in the inspector.
  4. Default image should be same size as widget.
  5. No JavaScript alert statements.
  6. Installing a widget in the Dashboard deletes it from the disk.
  7. Dashcode text parts don’t layout exactly the same on the iPad.
  8. Java and other plugins don’t work on the iPad.

 

14 responses to Creating Interactive Widgets for iBooks Author and the iPad – Part 1

  1. Thank you for this and Part 2. It is very helpful.

  2. Hi

    I’m working on a textbook and am trying to build in the ability for students to complete there assignments within the book and then email it to the marker.
    So I built a widget in Dashcode that takes the text input then sends it to the mailto: link which on my mac opens Mail with all the correct content, however on my iPad in iBooks the button does nothing at all.
    Any ideas on how to make it work or if its possible at this stage

    • Andrew,

      It appears the mailto: link is not supported by iBooks. I like the idea though, and it is an innovative use of the interactive widgets.

      What I would suggest is that you instead submit the data over HTTP. You could do this by setting up a RemObjects SDK server that accepted the assignment. That I know would work. Check out Part 3 for more information on that one.

      Another option that I haven’t tried is you could just do an HTTP Form submit. There are possible issue with that though too.

      Good luck and let me know how it turns out!

      -Jim

  3. another dimension to consider is to embed a html google doc inside the epub3

    this would allow interactivity
    and correction of the submitted work
    work offline
    and submit to a seciure account
    correction by teacher
    return of the marked up corrected work through google docs…

    its all html so a window embed ‘should be’possible..?

  4. Jim, Thanks for awesome explanation. I am currently creating an HTML widget and am trying to link to content within the book. In the epub2 format, I would just link to an xhtml page, but since the .iba format is flow-able, I’m not certain how to link to content with the book.

    Do you have any suggestions for that and perhaps a code sample?

  5. Hello,

    I would like to, first of all, thank you for the tutorial. It was very helpful.
    I just had a problem after previewing the widget on iBooks on my iPad: The “click” button disappears when I run the widget, though I manage to enter a text in the text Field, but the button disappears altogether. Do you know what could be the problem?

    Many thanks
    Rana

    • Rana,

      It sounds like you didn’t set the “Minimum Mac OS X version for this widget” to “10.4.0 (backwards-compatible)” on the Run and Share screen.

      -Jim

  6. Hey, I am having trouble with inserting a whiteboard type of thing into an ibook (I teach math and the students need something to work out their problems on). I tried just using an iframe inside a widget with http://www.awwapp.com. This works fine. However, I can’t get the widget to take up the whole screen. Any help would be appreciated.

    • Set the resolution of your default.png to 1024×768, or anything larger with a 4:3 aspect ratio. That should get you really close to a full screen widget when it is activated. Also remove the header and footer of the widget via the inspector.

      • thank you, I just figured that out. One other quick question. How do I get an image to display above the iframe? I want to display an image that shows the problem above the iframe when the widget is activated. When I try to do this, it messes with the iframe. Any help would be greatly appreciated!

  7. Thanks for your reply.
    In fact I have another question. I am trying to add a widget I created for our Geogebra application to iBook Author. My application is working on Dashcode (I can run it suing dashcode without errors). However, when I try to add it to iBook Author, I get the following error: The HTML Dashcode file GeoGebra.wdgt was not added. The file contains a media file that isn’t in a supported format.
    Note, that we are not using any media file, we are using HTML5, CSS and JavaScript. However, the JavaScript is produced through GWT (Google Web Toolkit).
    I was able to run another simple application created by GWT, but the GeoGebra application is not working.
    Do you have an idea what could be the problem?

    I could pass you a zip file with my code, if you think it is needed.

    Many thanks
    Rana