Archive for January, 2010

Persistentdata for beginners: RFile Logger

January 31, 2010

The RFile Logger component is a server for file logging used by the Test Execute Framework (TEF).

It is included in the sf codeline for dependency reasons – although it is logically part of TEF. The same RFile Logger code also exists in the Test Frameworks package (sftools/ana/testfw/testexecmdw/filelogger).

The API is provided by the RFileFlogger class, which is defined in test/rfilelogger.h. Macros are provided in test/rfileloggermacro.h to make it easier to introduce file logging for debug builds only. Both headers are platform headers.

Log messages can be written in a text or XML format. The logging output will contain a time stamp, thread id, severity, file name and the message. The file used for logging is specified by the call to RFileFlogger::CreateLog or the macro __FLOG_CREATE.

Coming up next…

Next in the Persistentdata for beginners series will be Feature Manager. It allows applications to determine at run-time whether hardware and software features are available to use.

Persistentdata for beginners: File Logger

January 29, 2010

File Logger provides logging to file for the purposes of debugging and field testing. It was developed as an internal logging framework for communication-related components, but over time it was used throughout the Symbian platform.

File Logger uses a client-server architecture, and provides a simple API to log to file as defined by the RFileLogger class (flogger.h). Client code should link to flogger.lib.

Static and non-static versions of the logging methods are provided for convenience. The static version is slower to use than the non-static version because it needs to reconnect to the server each time data is logged.

Logs can be enabled and disabled without having to re-compile by creating the directory you specify in your logging code under the C:\Logs directory (or on the emulator \epoc32\winscw\c\Logs). File Logger also provides a patchable constant to select logging using RDebug::Print.

The log output looks like this (excerpt from mmswatcher log):

29/01/2010 7:15:00 End of ConstructL
29/01/2010 7:15:48 RunL (iStatus = 0)
29/01/2010 7:15:48 DoRunL, state == 1
29/01/2010 7:15:48 StartupL
29/01/2010 7:15:48 Opening session

Further resources

Coming up next…
RFile Logger is next in the Persistentdata for beginners series.

Persistentdata for beginners: Event Logger

January 28, 2010

Event Logger stores events that are created by a number of operating system clients (e.g. events for incoming text messages generated by the SMS stack). These events are later retrieved by applications so the user can see a history of the events that have occurred on their device.

The Symbian/S60 Log application uses Event Logger to show a list of recent calls, packet data and other chargeable connections.

Event Logger has a client-server architecture and uses a DBMS database to manage the event log. The Event Logger API has been supported since S60 2nd Edition. It was previously known as log engine (logeng).

Event Logger API

Event Logger clients should link to logwrap.lib and logcli.lib.

API Purpose
CLogActive
(logwrap.h)
Abstract base class that defines common active object behaviour for event logger classes such as CLogViewRecent
CLogBase
(logwrap.h)
Used by client code to perform simple actions on the log database like adding a new event
CLogClient
(logcli.h)
The API to retrieve events from the log and administer the log database
CLogCntModel
(logcntmodel.h)
Base class for ECom plugin to resolve contact information from a phone number
CLogEvent
(logwrap.h)
Encapsulates the details of an event
CLogEventType
(logcli.h)
Event type, e.g. call, packet data, SMS, LBS transmit location etc.
CLogFilter
(logcli.h)
Specifies the conditions that events must satisfy to appear in a view
CLogFilterList
(logcli.h)
A collection of event view filters
CLogView
(logview.h)
Base class for log database views
CLogViewDuplicate
(logview.h)
A view of events that are duplicates of a specific event
CLogViewEvent
(logview.h)
A view of the event log
CLogViewRecent
(logview.h)
A view of recent events
CLogWrapper
(logwrap.h)
Wrapper around the event logger provided for compatibility reasons
MLogClientChangeObserver
(logclientchangeobserver.h)
Allows a client to observe specific change events (event log cleared, view refresh required)
MLogCntModel
(logcntmodel.h)
An interface to decouple Event Logger from the Contacts implementation
MLogViewChangeObserver
(logviewchangeobserver.h)
Allows a client to observe specific change events (event added/changed/deleted) within a given view
TLogConfig
(logcli.h)
Encapsulates Log Engine configuration data, such as the maximum number of events the event log holds
TLogSmsPduData
(logsmspdudata.h)
Used to store SMS PDU (protocol description unit) info in event data

Further resources

Coming up next…

File Logger is the next component in the Persistentdata for beginners series. It provides a logging API for debugging and other purposes.

Persistentdata for beginners: Store

January 27, 2010

Store provides the fundamental APIs for handling data on the Symbian platform – streams and stores.

Streams should be a familiar concept. They are part of the C++ standard library and Java’s I/O package. Objects can be externalized to a stream using the stream insertion operator and internalized from a stream using the stream extraction operator. The Symbian base classes that enable this behaviour are RReadStream and RWriteStream.

Stores are a collection (or network) of streams. Individual streams can be referenced by stream ID. Several types of stores derived from CStreamStore are provided to suit different needs.

Inheritance diagram for CStreamStore

Many applications use direct file stores (CDirectFileStore) to persist their preferences and DBMS uses the transactional properties of the permanent file store (CPermanentFileStore) to implement database transactions. However, stores do not have to be persistent (e.g. CBufStore).

(more…)

Persistentdata for beginners: SQLite 3 API

January 18, 2010

SQLite 3 API provides the standard SQLite C API on the Symbian platform. We introduced this component last year as a Symbian^3 feature to make it easier to port applications written in C (see bug 305).

If you’d like to learn more about SQLite, then I recommend the talk below. The lead author of SQLite, Dr Richard Hipp, explains all about SQLite, who uses it and what makes it unique. He describes the architecture and summarises the C and TCL interfaces. The talk is about 30 minutes long, followed by 15 minutes of questions and answers from the audience.

Introduction to SQLite – Google TechTalks May 31, 2006

Both Symbian SQL and SQLite 3 API use SQLite as their database engine.

As mentioned in the last post, Symbian SQL wraps up SQLite in a client-server architecture and offers an optimised integration of SQLite with the rest of the platform. This includes Symbian-specific features such as a Symbian C++ API, platform security, and performance/RAM usage optimisation.

The advantages of Symbian SQL make it the recommended choice for most applications, but the SQLite C API may be a better choice if you are porting code already using SQLite or developing new applications in C that will use SQLite. If you’re using Qt to write your application, you can use the QtSQL API (which has a driver for SQLite).

SQLite 3 API

You can find the sqlite3.h header file in \epoc32\include\stdapis. See Further resources below for links to API reference documentation. Client code should link to sqlite3.lib.

The sqlite3api component uses version 3.6.3 of SQLite. Note that the current build configuration disables virtual tables, SQLite loadable extensions and the ability to issue progress callbacks during long-running SQL statements.

Further resources

SQLite is comprehensively documented on the SQLite website (sqlite.org). Recommended pages for the C API are:

Coming up next…

The Store component will feature in the next part of the Persistentdata for beginners series.

Store has been a part of the Symbian platform since 1995. It provides essential APIs for data management – streams and stores.

Persistentdata for beginners: Symbian SQL

January 15, 2010

Dilbert.com

Symbian SQL is the client-server database that incorporates SQLite.

SQLite is a public-domain relational database library that is ACID compliant. Most readers of this blog will have used SQLite today. It is probably the most widely deployed SQL database engine in the world. It is included in Mozilla Firefox, Skype, PHP, iPhone, Android, McAfee anti-virus and many other products. It is included in Symbian devices starting from SymbianOS v9.3 (i.e. all Symbian PDKs).

Symbian SQL was created to provide a Symbian C++ API that would allow secure shared databases to be created according to a security policy and to introduce a shared memory cache between clients to reduce RAM usage. A description of the architecture is available in the Symbian SQL overview wiki page.

Symbian SQL offers three types of databases:

  1. Public – a database stored in the public area of the filesystem
  2. Private – a database stored in an executable’s private directory and only accessible to that executable
  3. Secure shared – a database stored in the Symbian SQL server’s private directory, shared with other clients according to a security policy defined by the executable which creates the database

Symbian SQL API

The Symbian SQL API is defined in the sqldb.h header file. Client code should link to sqldb.lib. The purpose of each class is summarised below:

API Purpose
RSqlDatabase A SQL database handle. It can be used to create, open and close a database, attach databases and execute SQL statements
RSqlBlobReadStream Handle to a binary or text blob for reading column data. It is used to transfer large binary or text column data from the SQL server to the client without the need to allocate memory to hold the entire column data. Available in Symbian^3 onwards
RSqlBlobWriteStream Handle to a binary or text blob for writing parameters. It is used to bind large binary or text column data without the need to allocate memory to hold the entire parameter data in the client. Available in Symbian^3 onwards
TSqlBlob Utility methods to read and write the contents of an entire blob in a single call. Available in Symbian^3 onwards
RSqlColumnReadStream Read stream for column data. This class is superceded by RSqlBlobReadStream in Symbian^3 which is more memory efficient
RSqlParamWriteStream Write stream for column data. This class is superceded by RSqlBlobWriteStream in Symbian^3 which is more memory efficient
RSqlSecurityPolicy Defines the security policies for a secure shared database
RSqlStatement A SQL statement used for data retrieval and manipulation
TSqlScalarFullSelectQuery An optimised API for the execution of SQL statements that return a single row consisting of a single column value
RSqlDatabase::TSize Class to hold a database size and database free space

Further resources

  • Symbian^3 reference documentation for Symbian SQL
  • The how to use Symbian SQL wiki page on developer.symbian.org describes the most important API methods and includes code examples
  • Guidance on how to optimise Symbian SQL usage is available on developer.symbian.org
  • The Symbian SQL wiki page on developer.symbian.org is the index for all wiki content relating to the Symbian SQL API and implementation
  • The Symbian Press booklet on Data Sharing and Persistence with Symbian C++ includes a section on Symbian SQL usage
  • Coming up next…

    The next part of the Persistentdata for beginners series will cover the SQLite 3 API component – the C API for SQLite.

  • Persistentdata for beginners: DBMS

    January 12, 2010

    The DBMS database engine was originally developed for portable computing devices in 1995.

    Designed with a small footprint in mind, it provides support for multiple database tables, transactions, shared concurrent access, SQL and C++ APIs. DBMS has limited functionality compared with “traditional” relational databases, but has good performance and efficient resource usage.

    It is used to store a variety of application and system data on the Symbian platform – contacts, log events, RSS subscriptions, Bluetooth settings and so on.

    DBMS can be used in two configurations: either as a client-server (shared) database or as a library linked against the application in the same process (private to the application).

    The DBMS API

    The DBMS API is powerful, but quite large. Here’s a summary of what the classes are for:

    API Purpose
    CDbColSet A set of column definitions for a table or rowset
    CDbKey Specifies a database table index
    CDbNames A collection of names returned by the DBMS API (used for database, table, and index names)
    CDbStrings A collection of strings passed from DBMS server to the client
    RDbColReadStream Stream class to allow column values to be read as stream data
    RDbColWriteStream Stream class to allow a long column to be streamed when inserting or updating rows in a rowset
    RDbDatabase Abstract class providing the functionality of a database
    RDbHandle<T> Internal class used by the DBMS implementation
    RDbHandleBase Internal class used by the DBMS implementation
    RDbIncremental Interface for performing long-running database operations in incremental steps
    RDbNamedDatabase Generic database implementation. Provides a concrete implementation of RDbDatabase
    RDbNotifier Provides notification of database changes to clients
    RDbRowConstraint Represents a pre-compiled SQL search-condition
    RDbRowSet Abstract base class providing cursor navigation, row retrieval and update behaviour of rowsets
    RDbs DBMS server session handle
    RDbStoreDatabase Store database implementation. Provides a concrete implementation of RDbDatabase
    RDbTable Provides access to table data as a rowset
    RDbUpdate Incremental interface to execute SQL data update
    RDbView Generates rowsets from an SQL query
    TDbBookmark Records the current position in a rowset
    TDbBookmark::TMark Stores a database bookmark
    TDbCol Defines a column in a table or rowset
    TDbColSetIter Iterator for a column set
    TDbKeyCol Defines a key column in an index
    TDbLookupKey Internal class used by the DBMS implementation
    TDbLookupKey::SColumn Internal class used by the DBMS implementation
    TDbQuery A SQL query combined with a text comparison mode
    TDbSeekKey Database key value
    TDbSeekMultiKey<S> Database multi-column key value
    TDbWindow Specifies the level of pre-evaluation for SQL query execution
    RDbDatabase::TSize Encapsulates database size and usage metrics

    Further resources

  • Symbian^3 reference documentation for DBMS
  • An article about DBMS from Dr Dobb’s Journal, Symbian Database Components: A small footprint DBMS for small footprint devices, written by Regan Coleman in 1996
  • A guide from Forum Nokia (S60 Platform: Using DBMS APIs v2.0) that demonstrates how to use the API with a simple example
  • Coming up next…
    The next part of the Persistentdata for beginners series will cover Symbian SQL, the client–server database incorporating SQLite.

    Persistentdata for beginners: Central Repository

    January 8, 2010

    The central repository component provides persistent storage of settings with support for transactions and change notification.

    It is the standard way to store settings and configuration elements on the Symbian platform. Examples of settings stored in central repository include the default starting view for the Calendar application, configuration settings for Location Based Services, and the access points needed to browse the Internet on your device.

    It is used throughout in the Symbian platform. Most packages rely on it. There are 665 users of centralrepository.dll in PDK 2.0.e according to the Agile Browser. This makes it one of the most used libraries in the platform (after efsrv, bafl, estor, and cone).

    Centralrepository.dll binary users

    Packages that use centralrepository.dll in PDK 2.0.e
    visualized by the Agile Browser 2.0


    Repositories and Settings

    Central Repository uses a client-server architecture to allow multiple processes to access settings and to enable central repository to enforce an access control policy (TSecurityPolicy) on behalf of its clients. The component name suggests that there might be a single repository file or database that holds all settings. In fact, the settings are held in a collection of repository files stored within the private directory of the central repository server (\private\10202be9).

    Each repository file is identified by UID. The filename of the repository file specifies the UID in hexadecimal (e.g. 101fd63d.txt). You can store up to 2^32 repositories, and each repository can contain up to 2^32 settings. Settings within a repository are identified by a 32-bit key. The type of settings that can be stored are signed integers, real numbers, binary data as 8-bit strings and Unicode strings. The maximum size of data that can be stored in a single setting is 2048 bytes or 1024 Unicode characters.

    Creating a repository

    It is not possible to create a repository using the API at run-time. You must initialise it first by supplying a text or compiled binary initialisation file at ROM build time if you’re a device creator or as part of your Software Install package file if you’re an after-market developer.

    The initialisation file is a UTF-16 format text file and specifies:

    • Header: File format version for central repository usage only
    • Owner (optional): SID of the application or component which is responsible for backing up the repository
    • Metadata (optional): Metadata for settings (e.g. choose if setting is backed-up and restored)
    • Access control: Defines a read and write policy for a repository, a range of settings or an individual setting using capabilities or SID.
    • Initial content: Specify what settings the repository will contain when first opened

    To improve performance when opening repositories, you can convert a text format initialisation file to a compiled binary format (.CRE) using the CentRepConv converter tool.

    Accessing Central Repository

    API (Header, Library) Purpose
    CRepository
    (centralrepository.h, centralrepository.lib)
    Class to access and use the repository
    CCenRepNotifyHandler (cenrepnotifyhandler.h, cenrepnotifhandler.lib) Class to start and stop notification of changes
    MCenRepNotifyHandlerCallback (cenrepnotifyhandler.h, cenrepnotifhandler.lib) Callback interface to handle the notification events

    The CRepository class provides access to a repository. The API includes methods to:

    • Create a new key (CRepository::Create)
    • Read a key (CRepository::Get)
    • Update a key (CRepository::Set)
    • Delete a key (CRepository::Delete)
    • Find a key (CRepository::FindL, FindEqL, FindNeqL)
    • Delete a key (CRepository::Delete)
    • Request notification (CRepository::NotifyRequest)
    • Start / commit a transaction
      (CRepository::StartTransaction, CRepository::CommitTransaction)

    CCenRepNotifyHandler is an active object wrapper around the single-shot notification API (CRepository::NotifyRequest). It provides an easy way for Central Repository clients to monitor changes to an individual key or the whole repository.

    To create a CCenRepNotifyHandler instance, the client needs to specify a connected CRepository session and callback class derived from MCenRepNotifyHandlerCallback. You use the CCenRepNotifyHandler::StartListeningL API to subscribe to notifications, and CCenRepNotifyHandler::StopListening to unsubscribe. Clients using CCenRepNotifyHandler derive from MCenRepNotifyHandlerCallback and implement the HandleNotify-methods that they are interested in and the HandleNotifyError method. For example, if you are listening for changes to an individual string key value you should implement HandleNotifyString and HandleNotifyError.

    Further Resources
    Links to further information about central repository:

    I haven’t covered all of the features offered by central repository in this post. If you have further questions, please contact the package team (see details how on my About page) or add a comment to this post.

    Coming up next…

    The next part of the Persistentdata for beginners series will cover DBMS: the original Symbian database engine.

    Persistentdata for beginners

    January 5, 2010

    Wordle image of persistentdata components

    Image created with wordle.net.

    This post is an index for a series of posts providing an introduction to the Persistent Data Services (persistentdata) package. The aim of the series is to provide a summary of the package APIs with some code examples. I will update this page with links to each post as they become available.

    The Persistent Data Services package provides robust, scalable and efficient persistent storage capabilities for the Symbian platform. It includes APIs for system and application data storage, event logging, feature management and instrumentation of code. It is part of the OS layer of the Symbian platform.

    The package includes the following collections and components:

    Persistent Storage collection

    Logging Services collection

    Feature Management collection

    Trace Services collection

    If you have particular questions or topics you would like me to cover in future posts, please let me know by adding a comment.

    The first part of this series (Central Repository) will be published soon.

    Persistentdata plans for 2010

    January 4, 2010

    2009 becomes 2010: Happy New Year! by Optical illusion

    2009 becomes 2010: Happy New Year!


    2010 is going to be an exciting year.

    We’ll be open-sourcing the persistentdata package under the Eclipse Public License, publishing a book on Symbian SQL and adding Open System Trace support to the entire package for easier diagnosis of runtime errors.

    This isn’t the full story though; I expect we will also have features contributed from the community during 2010. There is a lot of expertise outside of Nokia’s package team that I think we can benefit from.

    Hardening Symbian^2 and Symbian^3 will be our priority for the first half of this year. We have a couple of bugs remaining for Symbian^2 and we need to submit the final header relocations to complete our S^3 feature content (Bug 311).

    We’re planning Symbian^4 now. We currently have two features: OST support (Bug 312, 316) and a performance optimisation to central repository that will improve device boot time (Bug 1454). I will update the backlog with our planned contributions as more details are confirmed. You can see the schedule for all Symbian releases in the Platform Release Plan.

    In addition to open-sourcing the code, I think our development process will become more open this year. We will soon start making package-based deliveries to the Symbian Foundation instead of the centralised delivery being done at the moment. This will make it easier to provide more frequent deliveries with better release notes. I want to provide more visibility of our ongoing development work, so as a start I intend to set-up a feature codeline to support regular integrations of the latest SQLite code into Symbian SQL.

    I haven’t said much about the package code yet, so this month I’m going to write a series of posts to introduce the APIs provided by the Persistent Data Services package. I aim to provide a beginner’s guide to using the APIs we provide. Let me know if you have any topics you’d particularly like me to cover.

    Image “2009 becomes 2010: Happy New Year!” by Optical illusion used under a Creative Commons Attribution licence.