Tuesday, April 14, 2009

Writing a small Twitter client with Newspeak and Hopscotch

In this post I'm going to show a small Twitter client written using the Newspeak programming language and the Hopscotch framework.

As part of the process of exploring the Newspeak language here I'm going to focus on the Hopscotch UI framework. It will be used to present information from the Twitter REST API which is very easy to use.

The Hopscotch framework and IDE is described in the "Hopscotch: Towards User Interface Composition" paper by Vassili Bykov.

Code for this program was created using the Newspeak prototype from 2009-02-27.

The program

Here's a screenshot of the program:

screenshot fo the twitter client

The program is not a complete client, it just allows to post a new twit and to read the current time line of a the user.

The code from the previous post "Parsing JSON with Newspeak" is used to access the data provided by the Twitter services.

The code

The following screenshot shows the definition of the TwitterGUI class

The following nested classes provide the functionality for the client:

  1. TwitterClient: A class for using the Twitter REST API
  2. TwitPresenter,TwitSubject: The UI piece and information for a single twit
  3. TwitterMainPresenter,TwitterMainSubject: The UI piece and information for a the complete client

As described in the Hopscotch paper a pair of elements is required to create a UI piece. A subject which contains the information being presented and the presenter which defines the UI that shows it.

In the case of a single twit the data is transmitted from the service as a JSON document.

The following code shows an example of the JSON response of a single twit from the Twitter REST API:

"description":"Father, husband, friend, developer and late night programming language enthusiast.",
"name":"Luis Diego Fallas",
"created_at":"Mon Jul 28 13:51:55 +0000 2008",
"time_zone":"Central America",
"location":"Costa Rica",
"text":"Experimenting with Hopscotch in Newspeak",
"created_at":"Tue Apr 07 14:21:57 +0000 2009",

A TwitSubject represents one of these pieces of information.

class TwitSubject withTwit: twit images: images= Subject (
"Describe the class in this comment."
theTwit = twit.
theImages = images.
createPresenter = (
^TwitPresenter new subject: self.

The definition of the TwitPresenter looks like this:

class TwitPresenter = Presenter (
"Presenter for a single twit."
definition = (
^(padded:( column: {

(row: { link: (subject theTwit user screen_name) action: [] . }) color: ( Color veryVeryLightGray).
row: { image: (subject theImages at: (subject theTwit user profile_image_url)) .
blank: 4.
elastic: twitBody}.

}) with: {3. 3. 2. 2.}) .

The following screenshot shows an example of the previous definition.

Single twit

A special treatment need to be applied to the message since we want to be able to click on links or twitter user id's (not supported right now). The definition of twitBody shows this.

twitBody = (
| text result |
text: subject theTwit text.
((string: text contains: '@') or: [string: text contains: 'http'] )
ifTrue: [ result:: flow: ((text subStrings: {Character space}) collect: [:t | componentFor: t])]
ifFalse: [ result:: textDisplay: text ].

componentFor: s = (
(s includesSubString: '@')
ifTrue: [ result:: link: s action: [] ]
ifFalse: [
(string: s contains: 'http')
ifTrue: [ result:: link: s action: [openLink: s] ]
ifFalse:[ result:: label: s]

openLink:url = (
OSProcess command: (browser , ' ' , url).

What these methods do is to break the string of the message into words separated by spaces. If a word is an URL or and '@' character a link is created if not a 'label' is created . For the future this code needs to be improved with a better technique to identify urls.

The following code shows the definition of TwitterMainSubject:

class TwitterMainSubject user: userName password: pswd = Subject (
"Main subject."

user = userName.
password = pswd.
images = Dictionary new.
twitterClient = TwitterClient user: userName password:pswd.

createPresenter ^ = (
data:: twitterClient getFriendsTimeline.
^TwitterMainPresenter new subject: self

twits = (
data:: twitterClient getFriendsTimeline.
^data collect: [:t | TwitSubject withTwit: t images: imagesDictionary].

updateStatus: statusText = (
twitterClient updateStatus: statusText.

This class receives the used and password of the Twitter account. The TwitterClient class provides the access to the Twitter services.

Here's the definition of the TwitterMainPresenter class.

class TwitterMainPresenter = Presenter (
"Presenter for the main section of the GUI client."


definition ^ <Fragment> = (
^column: {
row: { label: 'What are you doing?'.} .
row: { elastic:twitEditor.} .
row: { getCharCountHolder.
button: 'Update' action:[updateStatus: (editor editedText asString)].
blank: 5.
button: 'Refresh' action:[twitsHolder refresh].
blank: 5.
row: {
blank: 5.
elastic:: getTwitsHolder

The getTwitsHolder method create an instance of an HolderComposer object that allows the content to be recalculated using the refresh method. Here's the definition:

getTwitsHolder = (
twitsHolder:: holder: [ list:: subject twits collect: [ :i | i presenter ] ].

Notice also that here we are requesting the list of twits to the subject, which calls the web service again getting new content.

Final words

The experience of using the Newspeak and the Hopscotch framework to create this program was very nice.

One thing that I need to find out is how to prevent the application from blocking when requesting the data from the services.

Code for this post can be found here.

Thursday, April 2, 2009

Parsing JSON with Newspeak

In this post I'm going to show a JSON parser written using the Newspeak parser combinator library.


Newspeak is a new programmming language. From its webpage http://newspeaklanguage.org/:

Newspeak is a new programming language in the tradition of Self and Smalltalk. Newspeak is highly dynamic and reflective - but designed to support modularity and security. It supports both object-oriented and functional programming.

In The Newspeak Programming Platform the authors give a nice introduction to the language and platform.

The first time I heard about Newspeak was by watching the Lang.NET 2008 symposium presentation video by Gilad Braha (available here). In this video, a nice parser combinator library is presented . This parser combinator library is described in the Executable Grammars in Newspeak paper by Gilad Bracha.

Code in this post was written using the Newspeak prototype released February 27 2009.


In order to learn about the language and platform I decided to create a little parser for JSON (Javascript Object Notation).

JSON is a simple data-interchange format defined in http://www.json.org/ .An example of it:

[{ "name": "Wiston Smith",
"description" :"Protagonist"},
{ "name": "Julia",
"description" :"Lover"},
{ "name": "O Brien",
"description" :"Goverment agent"}]

Parser structure

The parser is defined as a single class with a couple of nested classes. The following image shows the definition of JSON Parser in the Newspeak environment.

definition of the JSONParser class

As described in the "Modularity" section of the The Newspeak Programming Platform paper, top-level classes (in this case JSONParser) doesn't have access to its surrounding scope, it only has access to its own or inherited definitions. This is the reason why the JSONParser has the following definitions:

class JSONParser withParserLib: parserLibrary usingLib: platform = (
"Experiment for JSON parser based on the description from http://www.json.org/fatfree.html "
ExecutableGrammar = parserLibrary ExecutableGrammar.
CharParser = parserLibrary CharParser.
PredicateTokenParser = parserLibrary PredicateTokenParser.
Dictionary = platform Dictionary.
OrderedCollection = platform OrderedCollection.
Number = platform Number.

The "withParserLib: parserLibrary usingLib: platform" part defines parameters for the construction of JSONParser. These parameters are used to 'import' classes defined elsewhere.

The following code shows a way to create an instance of the JSONParser class:

|platform parser|
platform:: Platform new.
parser = (JSONParser withParserLib: (BlocklessCombinatorialParsing usingLib: platform) usingLib: platform).

The JSONParser nested classes are the following:

  1. CharExceptForParser which is a parser that accepts any character except for the one specified (this is for internal use)
  2. JSONGrammar The definition of the JSON grammar
  3. JSONGrammarWithAST which defines the way the AST is created
  4. JSONObject which is used in the representation of the JSON AST


The following code shows the JSON grammar defined using the parsing combinators:

class JSONGrammar = ExecutableGrammar (
"Experiment for JSON grammar based on the description from http://www.json.org/fatfree.html "
doubleQuote = (char: $").
backslash = (char: $\).
str = doubleQuote,((backslash, ( char: $" )) |
(backslash, ( char: $/ )) |
(backslash, backslash) |
(backslash, ( char: $r )) |
(backslash, ( char: $n )) |
(backslash, ( char: $t )) |
(charExceptFor: $")) star, doubleQuote.
string = tokenFor: str.

negSign = (char: $-).
plusSign = (char: $+).
digit = (charBetween: $0 and: $9).
dot = (char: $. ) .
num = negSign opt, digit, digit star, dot opt,digit star, ((char: $e) | (char: $E)) opt, (plusSign | negSign) opt,digit star.
number = tokenFor: num.

leftbrace = tokenFromChar: ${.
rightbrace =tokenFromChar: $}.
colon = tokenFromChar: $:.
comma = tokenFromChar: $,.
definition = string,colon,value.
obj = leftbrace, (definition starSeparatedBy: comma),rightbrace.
object = tokenFor: obj.

leftbracket = tokenFromChar: $[.
rightbracket = tokenFromChar: $].
arr = leftbracket, (value starSeparatedBy: comma), rightbracket.
array = tokenFor: arr.

ttrue = tokenFromSymbol: #true.
tfalse = tokenFromSymbol: #false.
null = tokenFromSymbol: #null.

value = string | number | object | array | ttrue | tfalse | null.


For more information on the how this library works, check the Executable Grammars in Newspeak paper.

AST construction

We need to define a way to represent the tree structure(AST) parsed by JSONGrammar. As described in the "Executable Grammars is Newspeak" paper, one of the nice things about Newspeak is that we don't have the modify the grammar definition to add AST construction code. We can do that by inheriting from the original grammar:

class JSONGrammarWithAST = JSONGrammar(
"Parses a JSON File and generates and Ast"

('as yet unclassified'
array = (
^super array wrapper: [:a | (a token at: 2) ].

null = (
^ super null wrapper: [:o | nil].

number = (
^super number wrapper: [:o | Number readFrom: (flattenCharCollectionToString: (o token)) ].

object = (
^super object wrapper:
[:obj | JSONObject withContent:
(Dictionary newFrom: ((obj token at: 2) collect: [:e | (e at: 1) -> (e at: 3)]))].

parse: input = (
^super value parse: input.

string = (
^super string wrapper:
[:t | flattenCollectedString: (t token at: 2)].

tfalse = (
^super tfalse wrapper: [:o | false].

ttrue = (
^super ttrue wrapper: [:o | true].


As shown here (omitting some method definitions) the arrays are converted to Ordered collections, the numbers,strings,booleans to its equivalents and JSON objects to instances of JSONObject(described below).


In order to make it easy to use a JSON object in Newspeak the JSONParser class was defined:

class JSONObject withContent: dContent = (
"Instances of this class represent JSON objects."
content = dContent.
('as yet unclassified'
doesNotUnderstand: message = (
| fieldName |
fieldName:: message selector string.
(fieldName beginsWith: 'json_')
ifTrue: [fieldName:: fieldName allButFirst: 5].
^content at: fieldName ifAbsent: [nil].


This class receives a Dicionary as parameter. This dictionary contains all the name/value pairs of the JSON object definition. We create a definition of the doesNotUnderstand method which as in Smalltalk is called when a message sent to an object doesn't have a explicit way to respond it. We take the name of the message being called and check it against the dictionary.

If the message is prefixed by 'json_', the string after it is used as the key in the dictionary. This is defined this way because JSONObject has definitions inherited from Object (such as 'name').

| parsed |
parsed:: parserWithAST
parse: (streamFromString: '[{ "name": "Wiston Smith",
"description" :"Protagonist"},
{ "name": "Julia",
"description" :"Lover"},
{ "name": "O Brien",
"description" :"Goverment agent"}]'
assert:[((parsed at: 2) description) = 'Lover'].
assert:[((parsed at: 3) json_name) = 'O Brien'].

In the following post I'm going to use this parser to explore the GUI library provided with Newspeak.

Code for this post can be found here.