Gilgamesh: Twitter over Bluetooth

 

Hey, looks, it is like Twitter… except without any Internet! The Gilgamesh App continues to evolves, and is now tagged at 0.0.4. You can unofficially find it on FDroid thought it might take a bit for the latest code to show up there.

device-2014-10-06-165429 device-2014-10-06-165405 device-2014-10-06-165421  device-2014-10-06-165447

 

The primary update is that I have made the user interface look somewhat like a real app. This includes an easy popup menu to reshare (“retweet”) messages, which helps expand the reach of any message, while the human aspect of it combats spam and false information.

I have also added the ability to send a direct (private) message to anyone easily, as well. The direct messages will be queued and stored until delivered, which means you can send a message when someone is out of range, and it will be delivered if they happen to come into range. Delivery is indicated by a ✓mark.

This is all still operating using the plain text Bluetooth Discovery name hack that I’ve been writing about here for the last week, or so. Any Bluetooth device be it a $1000 iPhone or a $10 Nokia can participate in this network simply by changing their Bluetooth name to have a space in front of it, and then writing their own status updates there. They can view all of the messages as well, by scanning for nearby devices. This Android app simply builds on top of that network to support private messages, and a persistent log of all status/names encountered.

Finally, just to review some of the privacy-enhancing aspects of this app:

  • If you just want to listen/consume information, you do not need to broadcast (make your device visible), making it very difficult to target your device
  • The passive broadcast/discovery conduit allows for people that do not know each other, to passively and async exchange information in public spaces, with their devices out of sight (in their pockets)
  • The asynchronous direct message features allows the exchange of messages in public places without any direct or visible interaction between parties
  • All data received in the app is stored in memory, and not permanently stored to the device. This means killing the app wipes all memory clean, leaving no trace behind.
  • All user identifiers are derived from the device’s ID, and though simple to remember, are not “friendly” in anyway that allows for easy social engineering impersonation attacks
  • If Bluetooth pairing is done between devices, an extra level of identity verification is provided, and an * is tagged to all identifiers when displayed to ensure this is who you think it is.
  • For Direct messages, any device which you have paired with, will use the Bluetooth “secure” socket connection mode, which provides a basic level of encryption and verification
  • No registration, real name, phone number or email is required, providing no link to any other identity
  • All resharing/re-broadcasting of information is powered by human minds and human hands, making it more difficult for any attacker to poison the information flow

As always, feedback on these, the code, the design and the concept, are welcome here, on github, diaspora, twitter, etc, etc, etc… see you around, and I really hope to find you on the Gilgamesh soon!

 

 

Another small step for a Tibetan keyboard on Android

There has been a lot of excitement this week about the robust support for the Tibetan written language in Apple’s iOS 4.2 for iPhone and iPad. This is a fantastic achievement that many contributed to, and that Apple should be loudly applauded for.

Unfortunately, the state of Tibetan on Android is still poor, but not hopeless. While Tom Meyer has provided a great starting point for rendering text properly, I still am not aware of any means for inputing Tibetan characters. With that in mind, I set out to investigate the ability to create a new Tibetan “Input Method” (as Google calls it) for Android, and quickly realized that one could just write a Language Pack add-on for the open-source AnySoftKeyboard project. This solution still requires you to root your phone and install the Dzongkha”རྫོང་ཁ font, but is still a step in the right direction!

You can find the open-source code for my new project, the Tibetan AnySoftKeyboard Language Pack on Github. If you would like to try it out, you must have a rooted device with the Dzongkha”རྫོང་ཁ font installed, then install the “AnySoftKeyboard” from the Android Market, then you can install the first test Add-on APK file, and the Tibetan option should come up in keyboard settings.

Below you can see a screenshot of the initial keyboard writing text into the OI Notepad app. It appears to be properly stacking characters as well, but I may be wrong. Also the current implementation does not yet support the SHIFT key or other modifiers. I would love to have some help setting up the rest of the QWERTY mappings in this XML file. Otherwise, any other comments, feedback, advice or pointers to other Android Tibetan keyboard work would be much appreciated!

Searching and Playing YouTube Videos using Appcelerator Titanium

This tutorial is part of content I am developed for the NYU ITP course I am teaching this semester, “Social Activism using Mobile Technology”. You can find the original document posted here. This code comes from the open-source project (of which I am the lead developer) located here: http://github.com/nysenatecio/NYSenateMobileApp. If you want to see this code “in action”, download the NYSenate Mobile app for iPhone, iPad or Android.
This tutorial will demonstrate using the Appcelerator Titanium Mobile API, a Javascript-based cross-mobile-platform toolkit, how to accomplish the following feats in a mobile app:
  • Search for video by channel name or keyword from the YouTube API
  • Use the Titanium HTTP client for async XML requests and response handling
  • Retrieve video results from YouTube and display thumbnail & text in a table
  • Create a WebView window and use it to display YouTube video player
  • Hand-off YouTube video links to the OS for external playback

All in all, following this tutorial, you should be able to perform the basic functions you would need to build an app that front-ended video content stored on YouTube.

First, we declare our variables:

//this is the main app window

var win = Titanium.UI.currentWindow;
//this creates a spinning widget we can display while the user waits

var toolActInd = Titanium.UI.createActivityIndicator();
//this is the table we will load videos into

var tableview;
//and the data array for the table

var data = [];
//the window and webview for displaying youtube player (iOS only)

var webModal;

var webModalView;
//stores the current link being displayed in the web view

var currentLink;
//this is the network request object

var xhr = Ti.Network.createHTTPClient();

Now you need to declare a function that knows how to play YouTube videos on iOS and Android. Every YouTube video has a GUID (globally unique id – or at least unique for YouTube) and a title. You pass those two values to this function and it will either launch the YouTube player directly (Android) or display a thumbnail with playbutton that the user can launch (iOS).

function playYouTube (vtitle, vguid)
{
if (Titanium.Platform.name == ‘iPhone OS’)
{
var ytVideoSrc = “http://www.youtube.com/v/” + vguid;
var thumbPlayer = ‘<html><head><style type=”text/css”> body { background-color: black;color: white;} </style></head><body style=”margin:0″><br/><br/><center><embed id=”yt” src=”‘ + ytVideoSrc + ‘” type=”application/x-shockwave-flash” width=”100%” height=”75%”></embed></center></body></html>’;

showHTMLContent(vtitle,’http://www.youtube.com/watch?v=’ + vguid,thumbPlayer);
}
else //on android
{
//this call to openURL hands off the link to the operating
//system, and starts any player that supports youtube.com
Titanium.Platform.openURL(‘http://www.youtube.com/watch?v=’ + vguid);
}
}

The reason you show an embedded YouTube thumbnail player on iOS, is that it will allow the video to play inside of the app without leaving the context of the app. This was more a problem on the non-multitasking iOS 3.x, and is still an issue on the iPad until iOS 4.2.

In the function above, there is a showHTMLContent() function call. This function is not built-in, and is just a way to simplify the common need to show some bits of HTML markup within your app. The arguments for the function are a title of the page, an optional URL to the source content of the page, and the direct HTML markup content to display.

function showHTMLContent(wTitle, wUrl, wHTMLContent)
{

//store the link for later use
currentLink = wUrl;

//create the window to hold the web view
webModal = Ti.UI.createWindow({});

//set the orientation modes for basically any which way
webModal.orientationModes = [
Titanium.UI.PORTRAIT,
Titanium.UI.LANDSCAPE_LEFT,
Titanium.UI.LANDSCAPE_RIGHT
];

//create the webview aka the embedded web browser (webkit/safari)
webModalView = Ti.UI.createWebView();
webModalView.scalesPageToFit = true;

//add the web video to the modal window
webModal.add(webModalView);

//set the title of the window
webModal.title = wTitle;

//if you are using a tab UI in the app, this will open the window
Titanium.UI.currentTab.open(webModal,{animated:true});

//set the HTML to display to the markup passed into the function
webModalView.html = wHTMLContent;

};

Great, so now we have the ability to display a YouTube player within our app using an embedded WebView. Pretty awesome so far, and hopefully you see how might use components like WebView for other mashups of native and web content in your apps.

Now we need to demonstrate how to get the data from YouTube on what videos are available. For this, we create another function called “doYouTubeSearch()”. This function takes two parameters: you can specify the channel name to retrieve videos from, or you can specify a search term, and you can combine these as well, to search videos from a specific channel only.

function doYouTubeSearch (channel, searchTerm)
{

//first show a “loading” spinning indicator to the user
toolActInd.message = ‘Loading videos…’;

win.setToolbar([toolActInd],{animated:true});

toolActInd.show();
//create the YouTube API search URL from the function parameters
var searchUrl = ‘http://gdata.youtube.com/feeds/api/videos?alt=rss&author=’ + escape(channel) + ‘&q=’ + escape(searchTerm) + “&orderby=published&max-results=25&v=2”;

//use the xhr http client object to do an HTTP GET request to the URL
xhr.open(“GET”,searchUrl);
xhr.send();
}

That was all pretty straightforward, right? You build up a URL, and you make the request using it for data from YouTube. Now, how you receive the response to that request is our next step. To do this, you must define an “onload” function for the ‘xhr’ object.

It is in this function that you will receive the data back from YouTube (usually in JSON or XML format), and you can process it to display in your app.

xhr.onload = function()
{
try
{

//the doc object holds the response structure

var doc;

//check whether the data coming back is in XML format or not

if (!this.responseXML)
{
//if not XML you have to convert it to XML
doc = Titanium.XML.parseString(this.responseText).documentElement;
}
else
{
//if it is XML, then just set the doc variable
doc = this.responseXML.documentElement;
}

//now we can easily get a list of items from teh results
var items = doc.getElementsByTagName(“item”);

//some simple variables for tracking the loop
var x = 0;
var c;

//now just loop through the response array to see what videos we have

for (c=0;c<items.length;c++)
{
//get the current item
var item = items.item(c);

//get the text for the video title tag using standard DOM XML calls
var title = item.getElementsByTagName(“title”).item(0).text;

//build up a summary string to display below the title
var summary = “”;
if (item.getElementsByTagName(“pubDate”))
{
summary = item.getElementsByTagName(“pubDate”).item(0).text;
}

//get the link to the youtube video
var link = “”;

if (item.getElementsByTagName(“link”))
{
link = item.getElementsByTagName(“link”).item(0).text;
}

//now here is where we perform a trick
//we find the GUID code from within the link b/c we know the link format
var guid = link.substring(link.indexOf(“?v=”)+3);
guid = guid.substring(0,guid.indexOf(“&”));

//now we can use that guid to load up a thumbnail image
var thumbnail = “http://i.ytimg.com/vi/” + guid + “/2.jpg”;

//okay we have all the data we need for that item
//now we need to create a row to add to the table in order to display it

//create the row item and set the height to 80 pixels
var row = Ti.UI.createTableViewRow({height:80});

//set parameters for the row so we can get the youtube data out later
row.url = link;
row.guid = guid;
row.videotitle = title;

//create a label for displaying the title and add it to the row
var labelTitle = Ti.UI.createLabel({
text:title,
left:105,
top:10,
height:40,
font:{fontSize:16}
});
row.add(labelTitle);

//create a label for the summary and add it to the row
var labelSummary = Ti.UI.createLabel({
text:summary,
left:105,
top:45,
font:{fontSize:12}
});
row.add(labelSummary);

//create an image from the thumbnail, and add it to the row
var img = Ti.UI.createImageView({
url:thumbnail,
left:0,
height:80,
width:100
});
row.add(img);

//add the row to the data array
data[x++] = row;

}

//if tableview has been created, reset the data on the table
//you can update data on the table multiple times
if (tableview)
{
tableview.setData(data);
}
else
{

//if table has not been created, build it up with the data array
tableview = Titanium.UI.createTableView({
data:data
});

//add the table to the current window for display
Titanium.UI.currentWindow.add(tableview);

//add a ‘click’ listener so that when someone taps on a row
//the video will be played using the function we defined earlier
tableview.addEventListener(‘click’,function(e)
{
playYouTube(e.row.videotitle,e.row.guid);
});

}
}
catch(E)
{
//if anything bad happens, show the error to the user and log it
Titanium.API.debug(E);
Titanium.UI.createAlertDialog({title:’NY Senate’, message:’No videos were found for this search.’}).show();

}

//hide the spinning ‘loading’ widget
toolActInd.hide();
win.setToolbar(null,{animated:true});
};

Okay, so that was a lot I know, but go back through it a few times, and you will see it is not so hard. First, we get XML back from YouTube. Then, we turn that XML into an array of items. Them we loop through those items and build up an array of rows. Then, we set the table with that array, and display the table. Finally, we handle the ‘click’ or touch events on the row, and display the YouTube player in the embedded webview. Ta-da! You now have a customizable YouTube search and player app.

Now here are three examples how you might kick off all this activity.

First, in this case, we are doing a search for any video in the ‘NYSenate’ YouTube channel.

doYouTubeSearch(‘NYSenate’,”);

In this example, we are searching all of YouTube for a “skateboard dog”.

doYouTubeSearch(”,’skateboard dog’);

Finally, in this example, we are

doYouTubeSearch(‘NYSenate’,’Brooklyn’);

In review, this lesson has showed you how to display HTML web content, how to create YouTube players embedded in that content, how to make HTTP requests to web services and APIs, how to parse the XML returned from those services, and how to display data in a Table format.

This example was built upon code that comes from the open-source project located here: http://github.com/nysenatecio/NYSenateMobileApp so go grab the code there and use it as a basis of your own app!

Mobile Apps under Duress: User Interface Guidelines

One of my students at ITP is working on a mobile application for the Android platform that will be used under duress and in generally stressful conditions. These situations might include documenting children at a refugee camp, capturing medical information in a remote clinic, or identifying victims after a national disaster.

I’d like to use this post to build up a list of useful guidelines for building mobile application user interfaces that can be effectively used during these situations. More specifically, with the capabilities of modern smartphones (large screen, capacitive/multi-touch touch, accelerometer, compass, camera and so on), what more can an application provide than just dumb entry forms and checkboxes.

Here’s my start… please add your own in the comments!

  1. Applications must be INSTANTLY responsive. Not only is there no time to lose in these situations, but the patience of the user will be at an all time low. Any data lookup should be cached, paged or otherwise optimized.
  2. Common tasks should be “shortcutable”… perhaps the user should be allowed to define their own shortcuts.
  3. Any queries or searches should be auto-magically remembered and available via dropdown so that the same text doesn’t have to be remembered multiple times.
  4. All actionable buttons/icons should be large… at least 64×64.
  5. Lists of selectable, pre-populated options should always be used instead of freeform text entry to improve accuracy of data entry. If freeform is required, suggestions for existing matches of data should be provided.
  6. Any network transmission of data or remote access should be done in the background without interrupting the work at hand. Again, remember the user should be expected to have ZERO patience.
  7. All color palettes should be HIGH contrast – the lighting situations may not be good AND the device screen brightness will most likely be set to very low in order to maintain the best battery life
  8. If the camera is expected to be used for image capture, remember that a 3 or 5 megapixel image can be quite large. Determine the need for resolution quality of the documented image and downsize that at capture time. Otherwise, loading, saving, and transmitting the captured photo could take up a lot of processor time and battery life.
  9. Make sure your UI works in both portrait and landscape modes… you just want the app to work no matter which orientation the device is being held.
  10. GPS – if you can use geolocation data to make the life of the user easier by prepopulating data or automatically geotagging items, then do it! However, they may need to turn off GPS in order to save battery life, so make sure to gracefully degrade.
  11. Any persisted data should be stored on the external SDCard storage so that it can easily be removed, backed up, read on a PC, etc… the phone might die, but you should be able to pop out the card and put it into a SDCard reader for any device to read. This may mean that instead of using the SQLite database on the device, you instead use an XML, JSON or CSV format on the card.
  12. If you need to record audio notes, a bluetooth or wired headset should be used. The built-in mics aren’t very good on most devices out today.
  13. Swiping, multi-touch or other gestures can be very natural and intuitive UI control mechanisms if used properly. If your applications lends itself to these, make sure you work closely with users to make sure they work… they should be almost natural for a user to do (like swiping photos left and right in a photo gallery) as opposed to some complex secret handshake.

Any more? Please add your ideas in the comments below…

ITP2800 – Week 5 – Building Mobile Apps

Homework for Week 5

  • Catch up on previous weeks’ readings and homework that you haven’t turned in yet
  • Continue to develop and plan your cause with a target on being able to concisely explain it for your midterm and the next Speed Geek!
  • Review the content presented in class and decide on the approach (server vs client or mixed) and the tools which you will utilize.
    • All: develop the user stories and basic mockups of your service – “iterate with pencil” before moving into fancier renderings
    • Server Developers: setup an IMified account and review the developer documentation
    • Client Developers: setup the Android SDK, then download and build the gReporter source code


Another exciting week at ITP. This week the classes was visited by Evan Henshaw-Plath aka Rabble, who both spoke of his personal experiences with mobile technology and activism, and also provided critical feedback to students on their projects during our first Speed Geek.

The second half of the class focused on preparing for the development of actual mobile applications and services. While students are free to choose any software or hardware approach they want, the two platforms that will be covered in class are IMified and Google Android.

Here is the audio of the second hour of class as I walk through the links and slide presentations embedded below:

Referenced links:

This first presentation provides an overview on approaching the development of mobile applications considering all of the unique factors of using a small, portable device on the go:

The second presentation uses the gReporter application to discuss the development of location and media-capture applications on the Google Android platform.

Finally, for beginning to design and mockup applications, here’s a list of solutions for the iPhone (thanks to TechJini blog)  that can also apply to Android:

  1. Use Interface Builder or Dashcode (Mac only)
  2. Use Balsamiq mockup tool (Mac, windows, linux) – http://www.balsamiq.com (http://www.balsamiq.com/blog/2009/03/01/iphone-controls-new-icons-and-much-more/)
  3. Use Omnigraffle (Mac only) and import an iPhone stencil – (http://www.omnigroup.com/applications/omnigraffle/download/ and http://www.graffletopia.com/search/iphone)
  4. Use paper and a stencil- http://www.designcommission.com/shop/iphone-stencil-kit/
  5. Use a pre-printed sketch paper – http://labs.boulevart.be/index.php/2008/06/05/sketch-paper-for-the-mobile-designer/
  6. Use photoshop and the iPhone PSD – http://www.teehanlax.com/blog/?p=1628
  7. Use Adobe Fireworks – http://blogs.adobe.com/fireworks/2008/08/iphone_gui_as_adobe_fireworks.html and http://www.building43.com/videos/2009/06/23/mockup-iphone-app-adobe-fireworks/
  8. Although not there yet, you can try http://iphonemockup.lkmc.ch/
  9. Use the stencil kit from Yahoo! which is available in a variety of formats – http://developer.yahoo.com/ypatterns/wireframes/
  10. Use the sketchbook available at http://www.mobilesketchbook.com/