## Canvas with javascript and coffee

Javascript is a bit awkward. The language itself is pretty powerful, but it has an obscure syntax. This week I decided to learn some coffee script which is a cure for javascript ugliness.

## Resources

Coffee has good learning resources, try:

official site

interactive ebook

and plenty of websites such as: this one where you are creating game of life on canvas.

## Beauty

Coffee is really pretty. Syntax is pretty simple and has lots of sugar. It uses whitespace semantically, similar to python. It make it more elegant and readable, however it can't be compressed easily. By compression I mean cluttered source code which is often used in web-development production code.

List comprehensions and classes syntax is also very nice. It speeds up development and makes code more readable.

## Availability

It can be embedded in browser directly or first compiled to js and then embedded as a regular js file.

Installing coffee is easy. You only need nodejs and npm.

For complete beginners this can be a nice starting point.

## Canvas

Canvas makes js and coffee real fun.

For the beginning I recommend this tutorial

My goal was to build a tetris on a canvas with coffescript. I made some progress and actually built a simplified version - which was at least resembling real tetris ;), but it needs polishing and second thought.

I learned that using classes in coffee script isn't usually the best idea. It can be tricky if you are used to classical concept of classes such as in C++ and python. Coffee is more functional oriented and it should be our approach.

## Alternatives

You should be aware that there are also different choices, which one is the best? It's hard to guess and even harder to try them all.

For example I recently stumbled upon gorilla-script.

It's easy to get overwhelmed by all these possibilities, but actually this abundance can help us to find a language which really suits us the best.

## Oh well, it should be latex.

I used this post: supporting latex in octopress

To enable this, I used kramdown and mathjax, look at changing markdown to kramdown http

Here is a very simple example of latex:

$$\xi = 15 \ \frac{d^2f}{dx^2} = 6xf - 88$$

I love writing articles in latex and after writing some recently, I thought about reposting it here. Now I have my tools. Stay tuned.

## Fitting text into categories

#### How to classify?

You can use various methods to classify objects. There are many tools in machine learning and natural language processing packages. But sometimes it's hard to find one, which suits your problem well. I wanted to write category classifier in scala, making it as simple as possible. I could use breeze, but at first glance it's quite overwhelming and documentation isn't really user friendly. (I love python scikit-learn, which is in my opinion the best machine learning library in the world)

I decided to implement it all by myself using only basic SDK of scala.

#### Main idea behind my script

In written language exist lots of common words such as: "the", "and", "this", etc. Specialised article or book contains lots of vocabulary connected to the field.

I did some web crawling to collect some articles with typical vocabulary for three basic fields: physics, electronics and mathematics. As a sample of normaly used vocabulary (not conected with any specialised field) I used some random articles from Guardian.

After collecting articles I could compute the most common worlds for each category. My classifying algorithm basically chooses a category, which vocabulary has the biggest intersection with given text, if intersection is very small it returns category "boring".

#### Implementation

{% codeblock splitContentToSentences - commonVocabulary.scala %} import java.io.RandomAccessFile import scala.collection.mutable.HashMap

def removePunctuation(text: String) = { val punct = ",.?;:!\"" text.toList.filterNot(char => punct contains char).mkString("") }

def getWordsFromRawText(text: String) = { removePunctuation(text).toLowerCase.split("\s+").toList }

def getAllWords(basePath: String) = { import java.io.File val baseDir = new File(basePath) val paths = baseDir.listFiles.toList.sorted var words = ListString for( path <- paths) { val raf = new RandomAccessFile(path, "r") val buff = new ArrayByte raf.readFully(buff) words = words ++: getWordsFromRawText(new String(buff)) } words }

def rankWords(words: List[String], normalVocabulary: Set[String]) = { var wordOccurences = new HashMapString, Int for(word <- words if !( normalVocabulary contains word) ){ if (wordOccurences contains word) { wordOccurences(word) += 1 } else wordOccurences(word) = 1 } wordOccurences.toList.sortBy(_._2).reverse }

val basePath = "articles/physics/" val normalVocabulary = getAllWords("articles/random/")

val commonNormalVoc = (for {word <- rankWords(normalVocabulary, Set("")).slice(0, 400)} yield word._1).toSet

val physicsVocabulary = getAllWords("articles/physics/") val commonPhysicsVoc = (for {word <- rankWords(physicsVocabulary, commonNormalVoc).slice(0, 400)} yield word._1).toSet

val electronicsVocabulary = getAllWords("articles/electronics/") val commonElectronicsVoc = (for {word <- rankWords(electronicsVocabulary, commonNormalVoc).slice(0, 400)} yield word._1).toSet

val mathVocabulary = getAllWords("articles/math/") val commonMathVoc = (for {word <- rankWords(mathVocabulary, commonNormalVoc).slice(0, 400)} yield word._1).toSet

val testArticleAboutPhysics = """ Thermodynamics is a branch of natural science concerned (...) or statistical mechanics, gave explanations of macroscopic thermodynamics by statistical predictions of the collective motion of particles based on the mechanics of their microscopic behavior. """

val testArticleAboutMath = """ Algebra can essentially be considered as doing computations (...) a polynomial in a single variable. """

val testArticleAboutElectronics = """ Wheatstone bridge (...) zero the voltage. """

val typicalGuardianArticle = """ The French president, (...) have legalised same-sex marriage. """

def matchCategory(text: String) = { val words = getWordsFromRawText(text).toSet val wordsCount = words.size val categories = HashMap("physics" -> (words intersect commonPhysicsVoc).size.toDouble / wordsCount * 100, "electronics" -> (words intersect commonElectronicsVoc).size.toDouble / wordsCount * 100, "math" -> (words intersect commonMathVoc).size.toDouble / wordsCount * 100).toList.sortBy(_._2).reverse

val winningCategory = if (categories(0)._2 > 10)
categories(0)
else
("boring", (words intersect commonNormalVoc).size.toDouble
/ wordsCount * 100)
println("Article matches category: " + winningCategory)
println(categories)
println("Similarity to normal vocabulary: " +
(words intersect commonNormalVoc).size.toDouble
/ wordsCount * 100)


}

I stored collected articles in directory "articles/{category}" and used them to generate vocabulary sets.

There are two important functions in this code: "rankWords" and "match category". Ranking is based of how many times given word occurs in the text. Matching categories using set intersections to estimate which category suits the best.

Here is the output of this script (test files were much longer):

{% codeblock commonVocabulary.scala output%} Testing article about physics: Article matches category: (physics,34.66666666666667) List((physics,34.66666666666667), (electronics,10.222222222222223), (math,8.0)) Similarity to normal vocabulary: 23.11111111111111

Testing article about math: Article matches category: (math,26.01880877742947) List((math,26.01880877742947), (physics,21.003134796238246), (electronics,13.793103448275861)) Similarity to normal vocabulary: 23.824451410658305

Testing article about electronics: Article matches category: (electronics,23.717948717948715) List((electronics,23.717948717948715), (physics,16.666666666666664), (math,14.102564102564102)) Similarity to normal vocabulary: 27.564102564102566

Testing typical guardian article: Article matches category: (boring,30.45977011494253) List((electronics,2.8735632183908044), (math,2.2988505747126435), (physics,2.2988505747126435)) Similarity to normal vocabulary: 30.45977011494253

It took some code, but it's actually pretty simple and works well. Scala offers really nice api to work with this kind of problems.

## The spring has come

This is a plan of the main campus of my university, Politechnika Warszawska, Warsaw University of Technology, WUT.

On the left is a building of faculty of Physics, where I go five times a week. Pretty nice place.

The bad news is that Warsaw hackerspace becomes hotter and hotter and our servers don't like that :|.

#### A... what?

Preon is a Java library which aims to provide a framework for dealing with binary encoded data. Repository

Actually reading binary files in Java isn't very comfortable. I love a python way to deal with binary files - struct

    import struct

#How exemplary data is packed
format = "<IIH2s"

#How long it is
how_many_bytes = calcsize(format)

#unpacking
something = struct.unpack(format, string_with_binary_data)

#packing
packed =  pack('llh0l', 1, 2, 3)


Nice and clean. What about Java? There are many ways to read binary files with Java. Almost all are very tedious. I decided to use preon. Here and there you can read some publications about preon.

#### How to use preon?

I think that from user point of view, the most important part of preon library are codecs.

//Create codec based on previously defined class
//Create buffer with data to be read
RandomAccessFile fp = new RandomAccessFile(file, "r");
fp.seek(offset);

import nl.flotsam.preon.annotation.BoundNumber;
//There is something like big and little endian (default is little)
import nl.flotsam.preon.buffer.ByteOrder;

//I use long with size 32 to represent unsigned 32 bit int
//Reading signed int is identical except from using int instead of long
@BoundNumber(size="32", byteOrder=ByteOrder.BigEndian)
long identifier;
@BoundNumber(size="32", byteOrder=ByteOrder.BigEndian)
@BoundNumber(size="32", byteOrder=ByteOrder.BigEndian)
long recordCount;

@Override
public String toString() {
+ headerLength + ", recordCount=" + recordCount + "]";
}

}


You can define byte order, size, type... and even lists with size based on value from one of fields. You can also add different methods to class (like toString).

I forgot to mention, Preon has lots of dependencies, you have to include them in your project or it won't work.

Not so hard, isn't it? And it has lot's more functionalities, but those above are the most common ones.

## GUI development in scalafx

#### Installation

There is a scalafx webpage and it has detailed tutorial how to install scalafx and get started. However it is actually even simples as described there. When you have working scala environment (eclipse with scala IDE for example) make sure that you use jdk from oracle (includes javafx by default and you need javafx to use scalafx) and just download scalafx jars and include them in your project.

#### Available learning resources

Creating gui with scalafx isn't hard when you know how to use it.

If you know javafx you've probably seen ProJavaFX book. Good news! There's a scalafx equivalent in source code

A better scalafx API overview is ensemble project.

#### When help from available examples is not enough

Asking google doesn't provide many helpful examples. I tried to resolve some of my problems, but I couldn't find right answer usually. Which is different from for example problems with java, because there are usually lots of questions with answers.

But... it's not so bad. Scalafx source code is available and well written and you can just skim it and get your answers or translate your problem to javafx problem and then look for answer in java community.

#### Mixing scalafx with javafx

I didn't all issues in my code(I have some problems with responsivness with observable buffers) but I can already tell that scalafx works well with javafx.

I wanted to add some warning dialogs into my scalafx application, but scalafx or javafx don't support it by default. I found dialogs library for javafx and tried to use it to show dialog in my scalafx app. Guess what? It worked and doing it was really simple.

{% codeblock splitContentToSentences - MainApp.scala %} import javafx.scene.control.Dialogs (...) new Button("Submit") { onAction = {e: ActionEvent => val emptyFields = getEmptyFields() if(!emptyFields.isEmpty) { Dialogs.showWarningDialog(stage, emptyFields.mkString(", ") + " cannot be empty.", "Book couldn't be added.", "Adding failure"); } } (...)

Here's the library: javafx-dialogs

and descriptive blog post about how to use it.

And I forgot to add - scalafx looks quite well (like javafx) opposed to scala swing which looks just horrible. Scala swing sounds nice, it's simple, is available by default, has scaladoc... but don't get deceived. Scala swing looks old, api makes strange suprises, and its development is a lie. Just use something else instead, for example scalafx.

## New love

#### Java was tiring and painful

I was developing a small project in Java. It was okay. Java isn't very difficult to write in. But I had some strange problems. Mainly with ide, libraries, maven, ant and so on.

I really hate build.xml. Oh, I do.

I wanted to do something stimulating and mix some scala code into my existing java project to make it more enjoable.

#### Mixing scala and java

I can't see you! It was the main reaction of my netbeans IDE. Actually mixing scala into existing java project isn't so easy. I installed special plugins for Netbeans but it didn't work, I got strange ant errors and spent a few hours reading about maven and building with ant.

It was too much for me. I returned to eclipse and installed ScalaIDE which is very nice, but still I don't know how to integrate there nicely a testing environment.

I created new Scala project because I read it's a better way than other way round (Scala in java) and added some of my existing java files.

I wanted to integrate a JavaFX, and then ScalaFX, but I had strange problems with those libraries.

#### Swinging with scala

Scala has swing library and its bindings are very intuitive and concise. So different from JavaFX!

I don't know it well yet, but I grasped some basics, write a really simple gui and it WORKED.

Simple German Swing tutorial

Working Sources

Enjoy.

#### Concise and functional scala

Scala on wiki

Scala website

On Scala website are links to learning materials, even free books like "Scala by example". There is also a brief tutorial (20 pages) which I recommend for the beginning, because it is a good overview. (Good start to fall in love with scala ;) )

Reading scala book is quite entertaining, I've never programmed a functional language before (not counting javascript and some lisp) and learning new concepts in such an elegant form as in scala is just a pleasure.

It's very expressive and still easy to use. Below you can see a small snippet of code I wrote today in the morning.

{% codeblock splitContentToSentences - SummaryTool.scala %} def splitContentToSentences(content: String) = { content.replaceAll("\n", ". ").split("\. ").filterNot(x => x.matches("\W+")).filterNot(x => x == "") }

SummaryTool.scala

#### Overall impression

Well, scala is cool. Rich type system, amazing api, conciseness make me a growing fan of scala. It's different from for example python, because it implements static typing which is usually a pain. Especially when writing with text editor not with speciallised IDE which analizes your code on the go. But in scala types are not just superfluous addition, they make code more understandable and functional and due to type inference you usually don't have to write them!

Now I'll return to scala book written by Oderski (main designer of scala). Only over 600 pages to go!

## Learning JavaFx2 - Part 1

#### Motivation

One of my projects has to written in Java or similar JVM language (not python sadly). It's a rich GUI application and I had to choose appropriate framework to develop user interface. After some research I found JavaFx.

I first wanted to do it in Groovy, dynamic language for JVM, but after I saw documentation of GroovyFx... I decided not to, because lots of subsection of this documentation was simply empty.

My project is not exceptionally ambitious and I wanted to do GUI as easily and fast and possible and then focus on major functionality of my app. I have developed some GUI's before in C++ and Python. And it was rather simple and intuitive process (I love Pyside!).

#### First impressions

I remember that I had some problems. I encountered very strange behavior of Intellij Ide, it was switching off after attempts to launch javafx project and I still don't know why. After some re-installations (SDK from Oracle and Netbeans IDE, which has default support for javafx) I started learning from the book: Pro JavaFX 2 which kindly provides source code.

I had custom installation of a development platform and I followed a tutorial, was it successful? Not. Some of the first examples from this book just doesn't compile on my computer. I lost lots of time, and found out that further examples works. WUT?

#### Philosophy

I think that's the main reason why JavaFX is so strange for me it's its javaland philosophy, which I'm not used to. I prefer thinking in terms of problem not in terms of framework. An approach where framework has so many solutions for me and is itself huge and has its own complicated philosophy is for me overwhelming.

Maybe I need more time for learning it and then using will be a piece of cake. I truly hope so.

#### Suprises

I read Pro JavaFx book to be able to build nice, simple and clean GUI. Documentation of Java libraries is usually not very friendly in my opinion, because it's usually automatically generated and there's too small signal to noise ratio whith all those classes and methods I usually doesn't need.

Funny thing is that one of the first application examples is a metronome, which uses a lots of transitions, which are quite funny in javafx. There a class and factory for everything. I usually think in terms on 2D canvas, but there it's implemented in different way. Nodes are binded to transitions and there is no trace of canvas.

Binding in JavaFx is a bit overwhelming. I've seen already at least 60 methods of public api connected with it. Nightmare! But it's quite useful when you get used to specific way of thinking about dynamic properties, which are observable, have event handlers and so on.

Chaining and builders are quite handy but they're not my favorite way of building objects, why not a wisely designed constructor? When construction of an object takes 50 lines of code, I simply get lost.

Here is an example of such code:

 java Some random node written in JavaFx



It's too long and it's ugly and my head's starting to ache.

#### Summary

I really start getting used to JavaFx peculiarities. Maybe I'll create beautiful GUI with it. I would really try but it may take some time. Stay tuned!

## Latex in use

### Problem - how to write a good report

Good report should look at least okay. My (and my partners) reports used to look quite ugly.

They weren't well formated and we know nearly nothing about typography and we are usually too busy to think about working on details for long hours. Instead we want to focus on computation and diagrams.

Writing lab reports is quite tedious in editors like MS Word or even Libre Writer (I'm a linux user, so I use the second one). There's to much clicking and searching for symbols etc.

### Solution

LaTeX

I heard from lots of people that latex is a nice tool for writing articles and publications. My partner had even some experience in it.

Two days ago I thought: "Why not try it now? With this small report?". I decided to do it and my partner agreed to help me.

But as you probably know, latex isn't so easy as libre writer...

### Learning Latex

It turns out to be lots of resources to learn latex. A good example is wikipedia wikibook on mathematics in LaTeX.

There are also many websites with tutorials and quite a lot of books.

In my opinion if you are a beginner, it's a good idea to install a latex ide. I tried a Texmaker. Texmaker is written in QT and available on all platforms. It even was in my repository (and I don't have ubuntu, but some odd, rare linux). It's got a good documentation and is pretty intuitive. My workflow in latex boosted.

### Problems

First problem I encountered was encoding. I wrote my report in Polish, a language which contains lots of strange characters i.e. "ę", "ó" and they were invisible in pdf I generated.

Quick search in google helped. There are packages for encoding! One with utf-8 helped.

\usepackage[utf8]{inputenc}


My second problem was inserting tables. I had to create a fancy table with spanning cells and wrapped content. Latex is not table friendly by default. I even tried to use my Texmaker ide, which have table wizard, but it was tedious and didn't give a desired effect. Happily, solution wasn't so hard - I found a detailed wikipedia wiki page on tables, which helped a lot!

It turns out that documentation is very good: arrows, symbols, more on symbols

All you need is in the internets.

Asking question is not so hard: tex forum because community is huge.

### Results

My report look quite impressive.

I learned lots of new things and I'm not tired of them and wanting more.

### Summary

Trying out LaTeX was a very good idea.

## Learning markdown

### Today morning markdown was for me the same thing as a markup.

Just a silly word. However it turns out to be a markup language which is very useful in blogging. I tried to learn some of it from wikipedia and the article which I found was pretty informative.

Try wikipedia article or this.

#### Why should I use it?

It's simple and easy to write. Writing < and all those html tags can be pretty tedious and you always can miss some, and usually output looks like a mess. At least for me. But there are solutions for this problem - markup languages!