Posts

 

Indiewebcamp and Web Intents

In June 2011 I made the journey to IndieWebcamp in Portland where Web Intents became quite a big topic. For an introduction to Web Intents read my last blog post “Web Intents – Gluing web functionally together

On the first day Chris Messina led a discussion that started with the architecture and focused in the possibilities of hijacking social buttons and turning them into Web Intents. Ben Ward carried on the discussion in another session.

The Web Intents could possibly fix the unwanted button issue and also allow large publishers and individuals to provide services on an equal footing. It’s this property which was so interesting to the people at IndieWebcamp.

Brainstorm user experience

At the end of IndieWebcamp Tantek Çelik and Erin Jo Richey brainstormed the current social sharing user experience and Web Intents. Erin posted a blog entry which has her thoughts on a possible user interface design. Tantek also posted a thoughtful piece that tries to give a historical context and future path for development through the studying of current use cases. There are some interesting comments on Goggle+ about Tantek’s post. Erin and Tantek have also put forward a talk for SXSWi on the subject.

Declarative mark-up for current social button patterns

Some of the discussion at IndieWebcamp was focused on whether we could use a browser extension to hijack current social buttons and overlay the Web Intents concept. The question asked was could we change a Facebook ‘Like’ button into a share button that used the registered Web Intents instead?

At the hacking session I started to build a prototype browser extension. It was primarily aimed at exploring the use of declarative mark-up as social button design pattern.

You need three elements to make Web Intents work the first is a browser based management interface. In this case I created a Firefox extension, a method to register the intent and a method to subscribe to the intent.

Web Intent registration

Ben Ward codified a hypothetical manifest structure for the Twitter Web Intents. It contains concepts for property name mapping and proprietary requirements. He based it on his experience of the Twitter Web Intents API. I have since added to the structure a little.

The registration of a service or group of services using JavaScript would typically look like this: />


function init(){
    window.intents.register({
        ‘vendor’: ‘Twitter’,
        ‘intents’: [
            {
                'name': "Tweet Link",
                'verb': 'share',
                'type': 'url',
                'required_params': ['url'],
                ‘url’: ‘http://twitter.com/share’},
            {
                ‘name’: "Retweet",
                ‘verb’: ‘share’,
                ‘type’: ‘status’,
                ‘required_params’: ['tweet_id'],
                ‘url’: ‘http://twitter.com/intent/retweet’,
                ‘param_map’: {‘object_id’: ‘tweet_id’}
            }
        ]
    });
}

window.addEventListener(‘load’, init, false);

Subscribing to a Web Intent

I built the subscription around the use of HTML5 ‘data-‘ attributes. This type of attribute can be used to provide all the additional information for a Web Intent. Its use would allow for progressive enhancement of a standard HTML link.

There are two distinct groups of attribute. The first group contains ‘data-verb’, ‘data-type’ and ‘data-url‘ which would be mandatory. Providing all the data needed to define a Web Intent.


Bookmark link

The ‘date-text’ attribute is an example of an optional data type. Supplying additional data would help the services to pre-fill forms that are presented to the user and improve the user experience.

Conceptually this design would use an agreed set of attributes for each verb/content-type combination, with a parameter mapping system so pre-existing URL end points would not have to be changed. We also added a proprietary naming scheme for extensions of optional data e.g. ‘data-v-twitter-via

The Firefox extension

Although only half finished it is possible to try out some of the prototype functionality. /> You can register a Web Intent

  • Allows for basic management through an interface
  • Finds the declarative HTML mark-up for subscriptions

To try this out, download the extension. It should install directly into Firefox then view the registration page and select “Allow” in the dialog

/>

There should be a small blue “in” icon in the Add-on bar at the bottom of the browser. Clicking this will bring up the management interface.

/>

Finally, navigating to the subscription page will show you an alert for each of the Web Intent subscriptions it discovers in the HTML. That’s as far as I got (it was only a few hours of hacking !).

Why have I not finished the prototype?

If we map out the approaches taken to implement the current social buttons you find 3 distinct models. They all use a URL endpoint for each service, but vary in how they exchange data.

  1. Uses the query string parameters which are pre or dynamically populated
  2. Extracts data from the referring URL. The service parses the HTML
  3. Exchanges data hidden behind a JavaScript interface

In practice you can see a combination of the above being used in a single implementation. Each model has different strengths and weaknesses. What is important is that all the current common use cases for social buttons are captured and that the Web Intent design has the widest coverage.

The design I envisaged passed data using the URL query string. This follows the design of Twitter’s Web Intents API and a lot of the other social button APIs. The Chrome and Mozilla teams are taking another approach still using a URL endpoint for each service but passing data as JSON objects through the context of the browser. This is arguably much more flexible.

Having read Tom Gibara’s post Reservations about the Web Intents system I do not agree with most of the points, but I can see the possible need to delegate URLs endpoints.

“…allows one web application to hijack an “intended” link to another. Example: an intent to view a Flickr photo by linking to its URL could be delegated to an application that displays Flickr photos in a way that is more to the users liking.”

Twitter has taken the approach of supporting just about every way of defining the data for a social button. They have also prioritised the order in which they discover it. The list below is taken from their button API documentation:

  1. Look for url in the share link query string.
  2. If not found, look for the data-url attribute of the Tweet Button anchor tag
  3. If not found, look for the rel=”canonical” link tag in the head of the document
  4. If not found use the URL of the webpage

If you added delegated URL endpoints to Twitter’s data entry this would be an extremely flexible solution. The question is whether this level of flexibility adds too much complexity. Only reviewing the use cases can give this answer.

I would like to spend some time considering this and as the Web Intent API is still in major flux while the Chrome and Mozilla teams align their work, I think this project is best left a while.

 

IMPORTANT /> The examples/code in this document are different to both Paul Kinlan webintents.org API and Mozilla Labs Web Activities API. In the past couple of months a lot has happened in this area and I now consider this work to be at a tangent to the main body of development taking place.

  • Data Portability
  • JavaScript
  • webintents

The page and the desktop - Skillswap talk

A couple of weeks ago I gave a talk at Brighton Skillswap called “The page and the desktop”. With some of the newer HTML5 APIs it is now possible to redesign how your web pages interact with the desktop. Web pages are all too often little islands that fail to interact well with the wider user interfaces of our devices. The talk tries to explain how the new Drag/Drop and File APIs could make web pages more equal in the world of applications.

The talked started with a series of demos that show what is currently possible. I have recreated the demos in a short screencast, hopefully it will give more context to the slides.

Example code

  • Data Portability
  • Events
  • JavaScript
  • html5

Small snippits of shinny ideas and tests

As I build projects I often try out new ideas, these sometimes get lost as I move on to other things. Codebits is a place for small snippits of shinny ideas and tests of all things web. I am going to use it to remind myself of how things work, but hopefully it will be helpful to others.

The first set of code sketches are all on the subject of copy and pasting. Over the next few weeks I will try and post up more stuff on ‘drag and drop’ and file APIs.

  • JavaScript

Semantic form mark-up discussion

As part of the mircoformats meet up in Brighton this week a group of us took some time to discuss the semantic mark-up of forms. This was based on my previous thoughts on the subject. We also look at this issue in light of Tantek’s proposal for the version 2 of microformats.

Please forgive that fact that I have not noted everyone’s name who took part. The main people contributing were; Aral Balkan, Glenn Jones, Jeremy Keith, Josh Emerson and Tantek Çelik. The session was broken down into 7 small discussions of specific problem areas. I have summarised below.

Using the class attribute and hierarchies

There was full agreement for the use of the className attribute instead of the form field name attribute. The name attribute can only take a single value and does not give enough flexibility to the author of a form as the name maybe dictated by the server side code.

For the current version of microformats, the use of hierarchical structures was agreed upon. The structures will be flattened in version 2 of microformats. In principle, form mark-up should follow the hierarchical structure of the microformat. /> Examples used for the discussion:

[sourcecode language="html"] />

/> /> />

/> /> />
/> [/sourcecode]

Defining types in the className attribute

There is a problem adding ‘type’ information i.e. home, work and pref to inputs that take telephone numbers or email addresses. There are many use cases where the author wants to define a specific ‘type’ of a property, such as a mobile telephone number, to a single input. To accommodate this we somehow have to add the ‘type’ metadata as well as the property name.

In the current version of microformats, most people at the meeting were happy with the type information being added into the className attribute. For version2 of microformts, Tantek suggested adding a prefix of ‘l’ standing for label.

There was some discussion of using the HTML label element to define the microformat property. But it was dropped as we could not think of an authoring use case and it would add complexity.

Tanteks microformats v2 idea: /> [sourcecode language="html"] /> /> /> [/sourcecode]

Example used for the discussion: /> [sourcecode language="html"] /> /> /> [/sourcecode]

Repeating microformat properties

This was generally agreed between everyone except Tantek. There are a large number of use cases where we find repeating patterns. These patterns allow a user to add multiples of one item. The most common example in hCard is the URL property. We discussed the use of a “repeatable” class that would cause an auto-fill application to duplicate the whole DOM node and append it as a sibling underneath the original.

The word “repeatable” was preferred over “repeat”. Although there are use cases to back up this type of functionally Tantek thought it broke the layers of abstraction and we could never be a 100% sure it would not damage the layout designed by the author. Tantek rejected this as a possible solution. No agreement/conclusions were reached about this point.

Example used for the discussion: /> [sourcecode language="html"] />

/> /> />
/> [/sourcecode]

Repeating microformat structures within one form

This item is similar to the previous item and as such everyone took a similar point of view. Jeremy believed that he could see good real world uses for this pattern of mark-up. Again “repeatable” would be a better className. Tantek’s previous objections were carried forward to this pattern.

Example used for the discussion: /> [sourcecode language="html"] />

/>
/> /> />
/> … />
/> [/sourcecode]

Logical operator “or”

There are times where an author may wish to use more than one property if it’s available. If we were to use this pattern the “or” is not required. Tantek pointed out that there is no order to the className attribute and we should not force one artificially to enable this feature.

Example used for the discussion: /> [sourcecode language="html"] /> /> /> /> /> [/sourcecode]

String concatenation

There are a number of circumstances where concatenating a plural microfromats property into a single string is required. The best example of this is a number of individual tags into a single string.

Jeremy suggested using the multiple attribute or a className multiple. As the multiple attribute can currently only be applied to a file input or a select element we will have to use the word ‘multiple’ in the class attribute. There was some discussion about dropping the formatting instructions.

Example used for the discussion: /> [sourcecode language="html"] /> /> [/sourcecode]

The group’s solution: /> [sourcecode language="html"] /> /> [/sourcecode]

Additional – Inputs inheriting parent className values

As we talked, it became apparent that it would be useful for a form element to inherit its parent className value. The form field would have to be the only form field of the parent. This type of structure would be required to ease the authoring of the ‘repeatable pattern’.

The group’s solution: /> [sourcecode language="html"] />

/> /> />

/> [/sourcecode]

Related microformat wiki documents

  • Microformats
  • forms

Experimental Firefox and Chrome extensions to copy and paste contacts

The extensions below allow you to copy, store and then later paste contacts into forms. They are proof of concept prototypes taken from a larger project. I have released the extensions now to help inform the discussion at the upcoming mircoformats meetup.

You need either Firefox 4.0+ or the latest Chrome browser.

/>Download Firefox extension (0.1.2 updated 16 Apr 2011)

/>Download Chrome extension (0.1.2 updated 16 Apr 2011)

2 minute video overview

How the extensions work

Copying contact details

Navigate to a page which contains a contact marked-up in hCard and right click on it. The context menu should then contain an item “Copy [Name] contact details”. Selecting this menu item will store the data in the browser.

Pasting the contact details

Navigate to a page which contains a form for contact details and right click on it. The context menu should then contain an item “Paste contacts…”. Choose a contact from the submenu and the extension will paste the data into the form for you.

Design considerations for the future

Issues of discoverability

The largest problem for the copy and paste of semantic data embedded in web pages is discoverability. It is difficult to indicate where and what data is available without over complicating the browser interface. I think the solution lays within joining the copy and paste functionality with other forms of interaction and/or visual flagging that will provide discovery.

Forms need to be marked-up semantically

At the moment most applications that auto fill web forms do not work very well. The common approach of “guessing” what data goes in which field is error-prone. Often the browsers/extensions use regular expressions to match the form field name.

Matching rule for a job title looks like this (?:occupation)|(?:job)|(?:title)

There are more sophisticated approaches which record matching rules from users, but even these are not perfect. What we really need is a way to semantically mark-up forms so there is little or no ambiguity. I have done some conceptual work on how microformats could be used to mark-up forms.

Labelling UX

Although the extensions currently work by parsing hCards I have used the terminology ‘Contacts’. This is for two reasons the first is that users do care about underlying schemas. The second is that a future interface design may abstract the data and display it differently to how it’s parsed from the page. For example, how do we deal with hResumes, do we just extract one contact or several based on work or educational history. If an hCard contains both a work and home address is it displayed as two contacts. Re-displaying the data in its raw form may not give the most intuitive interface.

Data source has to be comprehensive

Contacts found on the web are often just constructed of a few elements and not comprehensive enough to auto-fill a range of forms. The data may need to come from another more fully structured source than a web page for auto-fill to work relatively well. I also believe the user would need to be able to view what was to be copied, what is in the data store before making copy and paste choices.

  • Data Portability
  • Microformats
  • Projects

To quarters four hallo - 4000 year old hello world

The text below is “hello world” translated into Akkadian written in cuneiform script. Cuneiform is one of the earliest known forms of written expression, created on clay tablets using a blunt reed to create wedge shaped marks.

T
Q
4
H

I while ago Brain Suda did a great little post called “Welcome, the entire land” well worth a read. He was fascinated that Mike Stenhouse (London based UX designer) had a brief stint studying Egyptology. What better use for this cross-over skillset than to create a t-shirt that said, “hello world” in hieroglyphics. Ancient Egyptian does not quite have the vocabulary for “hello world” so after consulting the Egypt Exploration Society they came up with the translation “Welcome, the entire land”.

This project really made me smile and straight away I thought of a friend, Mark Weedon, who is an expert in Hittitology and Assyriology. This is the study of the archaeology and linguistics of ancient Mesopotamia.

The Akkadian translation he has provided me with is “to quarters four hallo”

  to quarter four hallo
Phonetic a-na kib-ra-tim ar-ba-im šul-mu-um

The grammar of Akkadian is not the same as modern English so it may seem reserved. The literal understanding “four quarters” is “river banks”, being the expression for the whole world, frequently used by kings summarising the extent of their conquests. The word “hallo” is a basic form of greeting, connected with health and hail. So it can be read as “to the world, hello”.

Mark did offer up a translation in a later form of cuneiform script “Greetings to the Universe/Totality” according to Mark this translation is from a later time when the old superlatives had to be surpassed, and kings would call themselves “king of the universe”! We went with the earlier form although I do like the king of the universe.

So in the spirit of Brains project I am just about to order myself a t-shirt. I have put the artwork and web fonts under a creative commons license, so anyone can download and use them. If you do make any interesting use of the artwork, please let me know. Maybe you also know an ancient language expert who could add another hello world translation.

I would like to thank Mark (Dr Mark Weedon, British Academy Post-Doctoral Fellow at SOAS) for the translation and his time hand drawing the letters forms for me.

  • akkadian
  • cuneiform

Proposal for using microformats to auto-fill forms

The current auto-fill browser extensions use regular expressions or previous user choices to work out how to automatically fill-in common registration forms. What we really need is a way to mark up forms semantically so that we can reliably auto-filled them using user selected structured data. Apart from obvious issues these applications also do not deal with repeating patterns very well, such as loading lists of friends or multiple email address.

As I worked on the draggables project and started to explore the possibilities of the W3C Contacts API this issue seems to be a hidden blocker to a number of interesting UX possibilities. I spent a week reviewing the common patterns of interaction and came up with a raw concept of how microformats could be extended to help with this problem. The proposal below was posted on the microformats wiki and I hope to carry forward the idea in the future.

Design goals for “input microformats”

Use the current microformat authoring conventions and schemas where possible. This goal should be understood in the light that most microformats are designed as hierarchical structures and whereas forms are not. Some breaking changes maybe required to deal with this fundamental difference, but these should be kept to a minimum.

Where possible any differences in authoring should be dealt with by creating a common superset of additional classes for auto-filling applications. This approach will limit the cognitive load on authors and allow the reuse of current parsers.

A primary design goal must be the consideration of i18n. This most effects dates and durations. The individual implementations may allow for language specific formatting, but it should not be part of this specification.

It should be reasonably easy for an author to add classes to a pre-existing form without having to change its data structure.

Auto-fill application vs generalist parsing documentation

Auto-filling a form and parsing its contents are two different operations, although they share the same conventions and schemeas. The discussion on how a microformat parser should extract values from form elements can be different to the needs of an application that can auto-fill a form using microformat data.

The input classname

To aid discovery and differentiate the use of microformats for the use of auto-fill applications we should use an input classname in conjunction with the root microformats classname. The current suggestion of appending the root classname with input ie vcard-input would break all current parsers. To add the demarcation we should use a new class input

Proposed markup:


[/sourcecode]

This would allow authors and applications to determine the intended use case i.e. the microformat mark-up is for form auto-fill. It would also be a simple task to update the current generalist parsers to ignore the mark-up if they were looking for content only. By default, most of the current parsers would already ignore an empty form marked-up with microformats as the required properties such as fn in hCard would be blank.

Form fields should use the classname attribute

It is tempting to consider using the form field name attribute as an alternative to the classname attribute, but this would break the current authoring conventions and all current parsers. We should confine the definitions of microformat to classname and rel attributes.

Proposed markup:

[sourcecode language="html"]

Text inputs and textareas

The mapping of content into text inputs and textareas is relatively straightforward, matches are made on the classname attribute and the whole value is used to fill in the form field.

Proposed markup:




Multiple form fields for plural microformat properties

Where a microformat property such as street-address in hCard can contain an array of values, these values will be added in order into the collection of form fields with the same classname.

Proposed markup:





Selects

Values can be passed into selects where a match can be found against the content or value of an option. The example below shows how a select could be used to define the type of a tel. The auto-fill application should check the content of the option first then its value attribute. Where a property such as a tel type is plural and the select element has the multiple attribute an auto-fill application should set multiple values in the select.

Proposed markup:


Checkbox inputs

Checkbox inputs can be used much like a multiple select. The values have to match the controlled vocabulary of the microfomat property being targeted.

Proposed markup:


Home Work etc…

Radio inputs

Radio inputs can be used to force a single selection from a controlled vocabulary. There are very few single value controlled vocabularies in the microformats schemas, but rating in hReview /> would be a good example.

Proposed markup:


5
4
3
etc…

Hidden input fields

In general, the use of hidden fields should be discouraged, but they are use useful for the post processing of language specific formatting. A good example of this would be hResumes use of duration to define how long someone has had a job. This is most often done with month/year drop down selects as on linkedin.com.

Proposed markup:



In the example above the duration value is correctly passing into a hidden input which will /> be submitted with the form allowing the page author to build custom i18n formatting through JavaScript. The alternative would be to display an ISO duration such as “P3Y4M”. This type of interaction design could be achieved using JavaScript hijacking techniques to aid clients that do not support scripts.

HTML5 input types

Although HTML5 has new semantic input types such as search and url their primary purpose is to describe data types rather than the context of data use. We may know that the input in the /> example below is a date, but we only know that it is a start date of an event by the use of the classname attribute dtstart.

Proposed markup:


For clarity and consistency in authoring rules, an auto-fill application should not try and imply context from the input type. All the non date and number based input types search, email, /> url and tel should be treated as if they were a simple text input.

The color input is not supported as it is not yet used as a data type in the current or proposed schemas

DateTime, Number and Range input types

The new HTML5 DateTime input types still do not give the data context, but they do inform us of specific formatting requirements. Where possible the auto-fill application should format the DateTime value as per the type. If the passed data does not contain the DateTime fragment required for the specific format the input should be left blank.

Proposed markup:


 1996-12-19
 1996-12
 17:39:57
 1996-12-19T16:39:57-08:00
 1996-12-19T16:39:57

The number and range inputs need to be type checked before data is past into the form fields. No invalid dates or numbers should be passed into the form fields.

Min and Max checking on form fields. If a min or max attribute exists for a number or datetime input the value being passed needs to be checked to make sure it is in range before the input is updated.

Repeating microformat properties

There are a number of plural properties in mircoformats that allow multiple values. In hCard the commonly used ones are tel, email and urls. To allow a form to extend to receive an unknown number of values auto-fill applications need to support a repeating pattern. This can be achieved with a new classname “repeat” which can be used in conjunction with a microformat property. The author needs to add an instructional classname to inform the application when to perform a repeat.

Proposed markup:


An auto-fill application working in a browser would duplicate the whole DOM node and append it as a sibling. All element ids and form field names would be appended with an index number to keep the HTML valid. The label for attribute would also be updated to any changed id reference.

Repeating microformat structures within one form

As well as repeating mircoformat properties whole mircoformats could be repeated within one form. Again, ids and form field names would be appended with an index number.



Defining types in the classname attribute

Most forms do not use hierarchical structures and as such the type/value structure used in microformats are less common in forms. Using the classname and type together allows the author to target a specific type/value i.e. a mobile telephone number or preferred email address.

Proposed markup:




String Concatenation

There are a number of circumstances where concatenating a plural microfromats property into a single string is required. The most common string concatenations involve a combination of spaces and/or comma’s. Auto-fill applications should concatenate three different patterns; comma-space-delimited, comma-delimited and space-delimited. These format operators have to be placed in the same classname attribute as the microformat property name. The concatenated string should be trimmed and there should be no trailing spaces or commas at the end of the string.

Proposed markup:



[/sourcecode]

Logical operators “or”

There are number of circumstances where an “or” operator would be useful. If a classname attribute with more than one microformat property and the “or “ operator the auto fill application will make a selection between the properties. The first non-null value will be used. Where the microformat property is a multiple value all the values of the first property are used before any subsequent properties.

Proposed markup:






Related microformat wiki documents

  • Identity
  • Microformats

Draggables

Just finished building the draggables library and uploaded it to its own site. Draggables is a JavaScript library that allows users to exchange data between web sites using drag and drop. It’s heavily based on the HTML5 Drag and Drop and File APIs. It allows data to be exchanged between unrelated sites and between the desktop and a browser.

http://draggables.com/ /> https://github.com/glennjones/draggables/

The project started a few months ago as a small UX experiment, I wanted to explore alternative way for a user to pass data between web sites. Having observed users connect to sites using OAuth without really understanding what exchange of data has taken place, I decided to investigate other metaphors/conventions that might be more transparent. Something that seem more “physical” or “tactile” an interaction, that represented movement and exchange.

Draggables started as a video of an idea and has matured a series of demos that other to explore the use of drag and drop to directly exchange data between web sites.

  • Data Portability
  • JavaScript
  • Projects
  • User Experience Design

Microformat Shiv

A light weight cross browser JavaScript Microformats parser.

Browsers should come with APIs to parse semantic content such as microformats, unfortunately this has yet to happen. I have just finished building microformat shiv which provides a simple to use, light-weight, cross-browser microformat API. It even works with modern mobile browsers and is backward compatible with Internet Explorer 6.

http://microformatshiv.com/ /> https://github.com/glennjones/microformat-shiv

The microformat shiv is based on the microformat API for Mozilla’s extensions. This is the same code base as the popular Operater plug-in. I have open sourced it under the Mozilla Public License and the source can be download from github.

  • JavaScript
  • Projects

Ident Engine - WebFinger support

Published: 21 February 2010 10:23

I have added WebFinger support to Ident Engine. You can now enter a Google gmail email address as a starting point for searching someone social graph. This only works for email accounts who’s users have published a public Google profile page.

Google’s profile pages are a good starting point to search the social graph. Google have design the profile entry to encourage user to build strong “elsewhere on the web” list of their other profiles pages across the web. These links are then marked-up with rel="me"

I used DeWitt Clinton WebFinger client which has a good JSON API.

  • Identity
  • Projects

Data formats:

API