Smart Phone
Smart Phone
Introduction
Remember when the technical preview for the Microsoft .NET Compact Framework was made available? I was really excited by the prospect of managed code running on my Compaq 3630. We have come a long way since that first release; from the Smart Device Extensions beta to its first 1.0 release for Pocket PC and Microsoft Windows CE. Now, the Compact Framework is included in the ROM for Pocket PC 2003. Our devices have shrunk again with Smartphone and with the release of the Smartphone 2003 SDK we can start writing applications in managed code for it. In this article I want to show you techniques and tips used in creating a user interface that is as close as possible to native Smartphone applications.
The source code you can download for this article implements and uses all the controls in this document, enabling you to rapidly adopt the guidelines covered.
Smart UI Basics
Let's start with a quick recap on good Smartphone user interface practice. If you already know the UI, you can skip to the next section. The Smartphone has no pointer/stylus, so we have to rely on a directional pad to navigate our forms. Generally we up/down navigate between fields and left/right navigate within a field. Once we take into account the window title and menubar, screen estate is limited to 176x180 pixels. An application should be designed to work at this resolution, but it should also scale out horizontally and scroll vertically for future proofing. For more information see the Positioning Controls section. The width doesn't leave much room for input fields, so labels identifying the fields are placed above the input controls. Standard convention is for these to be in bold and the input field in normal font. Input controls should be borderless; this is because Smartphone will automatically draw a border around the control that has focus (see Figure 1). If all our textboxes had borders we would have to look out for the cursor.
Figure 1. Input fields draw a border to signify focus. The form design can scroll vertically if we have too many controls to fit on the form; however, it should always fit horizontally. Additionally some of the larger controls, such as the multiline edit and listbox, have been put on a strict diet to ensure they can squeeze onto the user interface. They still retain their larger features; by selecting the action button they expand into full screen mode.
Figure 2. Controls on a diet, the expandable multiline editbox Traditional desktop and Pocket PC applications use buttons to enable users to carry out actions. Smartphone applications should never use buttons as this would require moving to focus on them and would not promote fast navigation or usage. The one place buttons can be seen is in a Web browser. Instead of using buttons to trigger actions, the phone has two hardware buttons below the screen, called softkeys, which map to a menubar containing up to two top-level menu items.
The Controls
Now that we have covered the basics let's look at the 14 user interface controls available in the .NET Compact Framework and how they map to a Smartphone 2003 project. Table 1. Comparison of the device controls and Smartphone 2003 Control Smartphone Usage Can be used to identify a field (should be formatted in bold) or to display text. Input field used to capture alphanumeric data. Can be multiline, see Figure 2. Used to run actions on the form.
Not supported on Smartphone 2003. See DataGrid section below. Input field to select an item. Condensed onto a single line for Smartphone, selecting the action key will expand out to fullscreen. Should be fullscreen.
Should be fullscreen.
Forms should only implement vertical scrolling. Horizontal ScrollBars should be used for non-input forms, such as displaying a large bitmap.
Not a visual control, used to raise events at specified intervals. Used to inform users of long running tasks. Not a visual control, used to store images used within the application.
This is a subset of the controls available to a Pocket PC or Windows CE project. Some of the missing controls such as the TabControl and ToolBar are not included since navigating these controls with the d-pad would not make sense.
The Label
Normally labels are coupled with an input field, providing identity for the input field. Figure 4 shows a label name identifying the textbox entry. These identifying labels are placed at the left side of the form and their default font is Nina, 11pt, bold. The label and textbox below have been placed at a Left value of 3, ensuring they both left align near the form edge.
Figure 4. The Label control At the top of the form is another label which is not coupled with an input field. In this case I have changed the font to Nina, 10pt, normal.
The TextBox
The textbox control is useful for text data entry. Remember, users won't write a bestseller using the keypad, so capture only what you need. By setting the Multiline property to true, the control can be expanded out to fullscreen for further input (see Figure 2). The OS provides this functionality along with the Done/Cancel softkeys. In addition we can also set what is the default input mode for the control, such as T9, multitap or numeric. See the Input Modes section later in this article.
Figure 5. MenuBars If you have more than 10 menu items under the right softkey, the OS will require the user to scroll up and down the pop-up, so it may be better to use nested sub-menus. This can lead to complex menu navigation, so it may be better still to divide the functionality across multiple forms.
Figure 6. CheckBox states Since a CheckBox control contains text to identify the control, the bold label identifier above the checkbox is not required.
Bitmaps
These have no visual differences to the Pocket PC. Bitmaps can be used to display image data, for example as you might receive from a Web service or they can be used as a custom drawing surface.
The Panel
Panel enables controls to be grouped together. See the Scrolling Forms section for an example of how a panel can be used.
DataGrid
If you look up the DataGrid in the Smartphone 2003 SDK you won't find it. That's because it is not listed as one of the supported managed controls. One of the differences between the Pocket PC and Smartphone platforms is RAM. A current typical Pocket PC device has 64MB, of which around a default 32MB is available to applications. My Smartphone has 16MB and the radio stack uses some of that. A large DataGrid is a serious RAM liability, partly since it will be backed by a large data source. With a significantly smaller working set performance will degrade, so this control is not supported.
ComboBox
On the desktop and Pocket PC the ComboBox control provides a drop-down list of items. This has been shrunk on the Smartphone to fit in the same space as the TextBox. It is also known as a Spinner control, see Figure 7.
Figure 7. Spinner control When the control has focus, the user selects left/right to move through the available items, selecting the action key expands the control out to reveal a full screen listbox, pre-selecting the current item. If the items do not fit, the OS automatically provides scrolling.
ListView
The ListView control provides the same functionality as the files and directories view in Windows Explorer, see Figures 8 and 9. It is usually used in conjunction with an ImageList control.
Figure 8. ListViewLarge Icon and List views If you look at all the internal applications on Smartphone, you will not find any user interfaces that use these ListView styles. Instead they use a customized ListView, also known as an owner drawn ListView. See the Owner-Drawn ListViews section for more information. Additionally it is possible to assign a CheckBox next to each ListView item. See the Missing Controls, Multi-Item Picker to see the user interface for this.
TreeView
By default the TreeView is sized to the full client area. It can be used with an ImageList control and for each node can provide a selected/un-selected image. See Figure 10.
ScrollBars
These are implemented in the same style as the Pocket PC. They do not provide automatic scrolling; instead you can programmatically set the scrollbar minimum/maximum and current position. An event can be raised when the position has changed, however it is up to the application to re-draw the form contents in the new position. See the Scrolling Forms section for an example.
Timer
This is not a visual control, but it must be used from within a windows form. It is used to generate an event at defined intervals.
ProgressBar
The ProgressBar control can be useful to update the user, the status of a long task. See Figure 11. Implementation is the same as the Pocket PC.
ImageList
This control holds a number of images which can be used throughout your application. The TreeView and ListView controls are designed to work directly with this control. You can add images using the design time editor; this automatically embeds the image as a resource so you don't have to distribute the image with your application. See Figure 12.
Figure 12. Adding images at design time You can also programmatically add images, for example if the images are dynamically loaded from a data store.
// Load the bitmap
System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(@"\MyImage.gif"); // Add bitmap to the image list this.imageList1.Images.Add(((System.Drawing.Image)bitmap));
Missing Controls
For a first release, the Compact Framework does a pretty good job at providing all the controls necessary to create great Smartphone applications. There are a few that are missing, and if you really miss them, they could be implemented as custom controls. Missing Controls Multi-Item Picker Native Support Enable the user to select multiple items, these look similar to the expandable edit field, with a comma delimited list of selected items. Selecting the action key expands to a full screen view showing all the available items.
It would be relatively easy to implement this control yourself. Create a custom control that traps the right or action key, then create a form with a ListView control in List mode with CheckBoxes. When the user selects Done, redraw the custom control accordingly. Provide a simplified input for either date or time.
This could be implemented as a custom control, by evaluating key strokes and limiting input.
Positioning Controls
The current Smartphone devices use a fixed screen resolution of 176x220 pixels. To future proof your applications it is worth building in the extra logic to ensure your applications scale correctly for different screen sizes. Since most entry dialogs will implement scrolling, see the Scrolling Forms section, we only need to worry about anchoring our input controls to the left and right of the form. The simplest way to achieve this is to set the input control widths in the form constructor.
this.textBox1.Width = this.ClientSize.Width-(2*this.textBox1.Left);
If your form contains a large number of label and input controls you may want to automatically size each input control within the form constructor.
// Set width for all input controls foreach (Control c in this.Controls) { if (c is Label || c is TextBox || c is CheckBox || c is ComboBox) { c.Width = this.ClientSize.Width-(2*c.Left)-this.vScrollBar1.Width; } }
If your control is a full screen control, such as a ListView or TreeView, it is recommended to set the size on the form constructor.
// Set the control to be full screen this.treeView1.Bounds = new Rectangle(0,0, this.ClientSize.Width, this.ClientSize.Height);
So now we've seen the controls available using the Compact Framework, let's look at bringing the controls together in a typical application flow. A great place for inspiration is the in-built applications on Smartphone. These have already had the effort put into the designing the user interface andin this caseplagiarism can ensure a familiar user experience. The Contacts application is a great example for a read/write style application. The first view is a ListView, providing a list of available items. Entering letters on the keypad automatically filters this view. Once the user finds the correct item, pressing the action key switches to the card deck view, this is a custom owner drawn ListView showing the detail for the item. Finally the user can select the left softkey and edit the item; in this case we have a standard entry form. Notice the focus is already on the Office address/Street field, which was focused on the previous view. See Figure 13.
Figure 13. List View, Card Deck View, and Edit View You will find above user interface pattern throughout the native applications on the phone. I would recommend looking through all the internal applications when starting your user interface design.
Input Modes
The first mobile phones with SMS capability required the user to type in each letter when sending a message, this was called multitap. Later models introduced a predictive input mechanism called T9, designed by Tegic Corporation. Instead of having to hit a key on the alphanumeric keypad multiple times, the word is predicted as you type. Smartphone provides four different input modes, in addition to multitap and T9, the device also has a numeric setting for numbers only and a system wide multitap or T9 setting, which remembers the last input mode. The Compact Framework does not provide a class to change the input mode, so we need to look at how native applications set this. The Smartphone API provides the message EM_SETINPUTMODE which can be sent to a windows handle to set the input mode, so we
need to use native interop to achieve this. Notice the title bar displays the current input mode on the right, next to the signal strength indicator, see Figure 14.
This code enables you to easily change the input modes on the device.
SetInputMode(myTextBox, InputMode.T9);
Warning The use of HWNDs from within the .NET Compact Framework is unsupported and should be thoroughly tested within your application. Due to how the runtime internally manages these, use of HWNDs may not work with future releases or could have breaking consequences if you do a lot of manipulation with them. Note T9 support is not included in the emulator. If you set this mode on the emulator, the input mode will switch to multitap.
Splash Screens
Many applications implement a splash screen on start up. This is really easy to achieve on the Smartphone. Create a new form without a MenuBar and change the WindowState property to Maximized, this ensures the form fills the entire screen. Finally add controls and any graphics to your splash screen. See Figure 15.
Figure 15. Full screen splash screen When designing your splash screen, take into account positioning of controls. For example if you display a bitmap, it may be best to position the bitmap in the center of the form, or for labels set the control width to fit the screen, to ensure compatibility with any future screen resolutions. Since the runtime resolution may be different to the design time form size, we need to use check the ClientSize of the form to find out the true full screen resolution of the display. Then within our splash form constructor we can find out the screen dimensions and position our controls.
// Set the form to full screen. No titlebar or menu.
this.WindowState = FormWindowState.Maximized; // Set width for all input controls foreach (Control c in this.Controls) { c.Width = this.ClientSize.Width-(2*c.Left); }
If necessary we can anchor controls at the bottom of the screen using the following code.
// We want the statusLabel, separator and copyrightLabel to be anchored to the bottom of the form // Find the difference between the lowest control (statusLablel) and the height of the form int yPos = this.ClientSize.Height - this.statusLabel.Top this.statusLabel.Height; // Move each control downwards this.statusLabel.Top += yPos; this.separator.Top += yPos; this.copyrightLabel.Top += yPos;
Scrolling Forms
So far all the forms we have looked at fit to screen. We have seen some controls can provide automatic scrolling, such as the ListView and the Expandable Edit and Spinners when in full screen mode. In this section let's look at how we can create our own scrolling forms. In the native world creating a scrolling dialog was simple. You would simply extend the dialog beyond the screen size and set the vertical style. The Smartphone OS would then automatically draw and size the scrollbar depending on the number of focusable controls on the form. The initial release of the Compact Framework does not provide automatic scrolling although we can still implement this using a panel and vertical scrollbar. First extend the form vertically to hold all the controls. Add a VScrollBar to the top right of the form and a panel to occupy the remaining space. Then add all the controls within the panel control.
Figure 16. Designing scrolling forms When the form is instantiated the scrollbar dimensions are set.
// Set the scrollbar dimensions this.vScrollBar1.Height = this.ClientSize.Height; this.vScrollBar1.Left = this.Width-this.vScrollBar1.Width; this.vScrollBar1.Minimum = 0; this.vScrollBar1.Maximum = this.panel1.Height-this.ClientSize.Height+5;
It is also good practice to programmatically set the width for each control within the panel, for future screen dimensions.
// Set width for all input controls foreach (Control c in this.panel1.Controls) { if (c is Label || c is TextBox || c is CheckBox || c is ComboBox) { c.Width = this.ClientSize.Width-(2*c.Left)-this.vScrollBar1.Width; } }
On a native Smartphone application, when the user moves onto the next focusable control, the OS checks if the control is within the client area. If it is outside the area then the form will scroll up or down accordingly. When the form scrolls upwards, the application should scroll to the label identifier for the field, if one exists. See Figure 17.
Figure 17. Scrolling upwards When scrolling downwards the application needs to ensure the target control is just within view of the client area, see Figure 18.
Figure 18. Scrolling downwards The following function can be used to facilitate this on NETCF.
/// <summary> /// Sets the scrollbar and panel position /// </summary> /// <param name="topSender">The identifier label for the control, or the control if no label exists</param> /// <param name="bottomSender">The control with focus</param> void SetScrollPosition(object topSender, object bottomSender) { //get bounds of controls to focus on int top = ((Control)topSender).Top; int bottom = ((Control)bottomSender).Bottom; int height = bottom - top; //get scroll position int pos = this.vScrollBar1.Value; //check if control is within view if (pos < top && bottom < (pos+this.ClientSize.Height)) { //do nothing return; } //check if control is above view if (bottom < pos + height) {
//scroll up to view, ensuring the topSender is first visible on form this.vScrollBar1.Value = top; } //check if control is below view if (bottom > (pos+this.ClientSize.Height)) { //scroll down to view, ensuring the bottomSender is last visible on form this.vScrollBar1.Value = bottom - this.ClientSize.Height + 5; } // Set the panel position on the form, to redraw the application this.panel1.Top = -this.vScrollBar1.Value; return; }
We can then use this function to set the visible position, when focus is received.
private void MyEditControl_GotFocus(object sender, System.EventArgs e) { // If scrolling upwards, ensure MyLabel is visible // If scrolling downards, ensure the sender (MyEditControl) is visible SetScrollPosition(MyLabel, sender); }
Owner-Drawn ListViews
In the final section I want to cover the ListView control, but not the same ListView we covered earlier. Let's take the native Inbox application, see Figure 19.
Figure 19. Inbox ListView Like the Contacts application we saw in the Card Deck View, this is a custom owner-drawn ListView. Instead of adding ListView items to draw, we take over the paint method and draw our own items. Let's look at implementing an owner-drawn ListView in the similar style as the Inbox application. All owner-drawn ListViews have common functionality, like the ability to navigate and hold items of data. This would make sense to implement this in a base class, from which we can inherit for our custom ListViews. See Figure 20.
Figure 20. Base ListView functionality When we create our own ListView based on the OwnerDrawnList class we only need to implement two methods.
OnPaint OnPaintBackground
Notice the base class has a member, offScreen. This is a Bitmap object which is used as a custom drawing surface within our derived class. The items member is an ArrayList, which can be used to hold the ListView data. For this example I have created a simple MailItem class for each ListView item.
class MailItem { public MailIcon icon;
public string sender; public string subject; public MailItem(MailIcon Icon, string Sender, string Subject) { this.icon = Icon; this.sender = Sender; this.subject = Subject; } } enum MailIcon { unopened, opened }
OnPaint
Within this method we can implement the custom drawing onto the offScreen Bitmap object. The example below will draw a mail item, consisting of an image to show if the mail has been read or unread, the sender and subject line. If the mail is unread all the text is in bold font.
protected override void OnPaint(PaintEventArgs e) { // Declare vars Font font; Color fontColor; string bmpName; // Get graphics object from bitmap. Graphics gOffScreen = Graphics.FromImage(this.OffScreen); // Set background color gOffScreen.FillRectangle(new SolidBrush(this.BackColor), this.ClientRectangle); // Set the y pos of the current item int itemTop = 0; // Draw the visible items. for(int n = this.VScrollBar.Value; n <= this.VScrollBar.Value + DrawCount; n++) { // Draw the selected item to appear highlighted if(n == this.SelectedIndex) { gOffScreen.FillRectangle(new SolidBrush(SystemColors.Highlight), 0, itemTop, // If the scroll bar is visible, subtract the scrollbar width this.ClientSize.Width - (this.VScrollBar.Visible ? this.VScrollBar.Width : 0), this.ItemHeight); fontColor = CalcTextColor(SystemColors.Highlight); } else fontColor = this.ForeColor;
// Draw a gray separator for each item gOffScreen.DrawLine(new Pen(Color.DarkGray), 1, itemTop+this.ItemHeight, this.ClientSize.Width - (this.VScrollBar.Visible ? this.VScrollBar.Width : 2), itemTop+this.ItemHeight); // Get the current MailItem MailItem lvi = (MailItem)this.Items[n]; // Set font and image pending mail if (lvi.icon == MailIcon.unopened) font = new Font(this.Font.Name, bmpName = "SmartUI.unread.bmp"; } else { font = new Font(this.Font.Name, bmpName = "SmartUI.read.bmp"; } state { 10, FontStyle.Bold);
10, FontStyle.Regular);
// Load image Bitmap bmp = new Bitmap(Assembly.GetExecutingAssembly().GetManifest ResourceStream(bmpName)); // To draw a transparent image, we need to set the transparent color. in our case red ImageAttributes ia = new ImageAttributes(); ia.SetColorKey(Color.Red, Color.Red); // Set the image rectangle Rectangle imgRect = new Rectangle(Column1Left, itemTop+3, bmp.Width, bmp.Height); // Draw the image gOffScreen.DrawImage(bmp, imgRect, 0, 0, bmp.Width, bmp.Height, GraphicsUnit.Pixel, ia); // Draw the mail sender gOffScreen.DrawString(lvi.sender, font, new SolidBrush(fontColor), Column2Left, itemTop); // Draw the mail subject gOffScreen.DrawString(lvi.subject, font, new SolidBrush(fontColor), Column2Left, itemTop + (ItemHeight/2)); // Cleanup font.Dispose(); bmp.Dispose(); // Set the next item top to move down the item height itemTop += this.ItemHeight; } // Now draw the visible list box e.Graphics.DrawImage(this.OffScreen, 0, 0); gOffScreen.Dispose();
The read.bmp image needs to be drawn with a transparent background, since it is not rectangular, see Figure 21. This is achieved by filling the transparent area of the bitmap with a color key, a color not used elsewhere in the bitmap. In this example I have used red. The ImageAttributes class allows you to set the color key, which is then passed as a parameter to the DrawImage method, this then ignores the pixels that match the color key.
OnPaintBackground
This method needs to be over ridden to prevent the OS from repainting the background each time the user navigates the ListView, causing flicker. If you don't need a background then just override this method but provide no implementation.
When the control is drawn, the overridden OnPaint method is called drawing our OwnerDrawn ListView, see Figure 21.
Capturing Input
Once the user has found the correct ListView item, it is good practice to allow them to select it by pressing the action key on the d-pad and by a softkey. We can do this by overriding the OnKeyDown method within our derived ListView.
// Check if the user presses the action key protected override void OnKeyDown(KeyEventArgs e) { switch(e.KeyCode) { case Keys.Return: MessageBox.Show("You selected item " + this.SelectedIndex.ToString(), "Item selected", MessageBoxButtons.OK, MessageBoxIcon.Asterisk, MessageBoxDefaultButton.Button1); break; } base.OnKeyDown(e); }