Testable triggers for SouthPaw Tactic.
The last six months have been spent working on a project based around the recently open-sourced Tactic. Â One issue we came up against during the project was being able to easily develop and test custom triggers.
Custom triggers are simply python scripts that have a couple of extra magic variables (server, input), ignoring the fact that the input variable is actually shadowing a builtin the fact we have magic variables makes this a little more difficult there is however a rather simple approach that gets around this problem.
def main(server, input_data): pass if __name__ == '__builtin__': main(server, input) elif __name__ == '__main__': pass # Create server instance and input dictionary for testing.
This of course is very similar to the standard way of making a module either usable within a program or from the command line.
In addition to this code we have developed a library for importing and exporting triggers, scripts etc from Tactic which we are likely to release back to the community in the near future.
PyCharm and Nosetests
I have to say that this is really really un-obvious, but after a lot of trying (or a wasted hour) here is the solution.
I have a folder in my Denim project called tests (I’m sure you can guess what is in that folder…) from the command line running all the tests in that folder is pretty straight forward, point nose at the folder and my tests execute.
So I add a nosetests run configuration in PyCharm point it at my test folder and:
Ran 0 tests in 0.000s
Ok not quite what I was expecting, the PyCharm docs are similarly unhelpful, even mentioning that I need to fill out a pattern field that isn’t even on the form (looks a lot like documentation was cloned from Unittest documentation).
Anyway turns out the solution is really simple, check the Params option and enter –all-modules tests now all complete.
Happy testing! (you do write tests don’t you đ )
Google Protocol Buffers and BlackBerry 10
A brief HowTo on compiling the protobuf library for use in a BB10 application.
I will be assuming you are using the QNX development environment provided by RIM on GNU/Linux (in my case Ubuntu 12.04) and you have the following packages installed:
- build-essential
- protobuf-compiler
Step 1 – Preparation
Activate the QNX environment for the version of the BlackBerry 10 NDK you require.
. ./bbndk-env_10_1_0_1720.sh
Step 2 – Downloading
Download and extract the protocol buffers library from Google Code in my case I downloaded version 2.4.1 to match the version packaged on Ubuntu 12.04.
Step 3 – Compilation
This step requires configuring a make file for each target that you want to build for. Both armle (for device) and x86 (for the simulator), if you do development directly onto a device you can skip the second compile.
Use the supplied configure script to generate a make file (for device/armle):
CXX=${QNX_HOST}/usr/bin/ntoarmv7-c++ LD=${QNX_HOST}/usr/bin/ntoarmv7-ld ./configure --host=armle --disable-protoc --with-protoc=protoc --prefix=/PATH/TO/INSTALL
This command specifies which compiler and linker to use, specifies the host as being armle (thereby enabling cross compilation), disables protoc from being built, specifies that the system protoc should be used to generate any code and finally specifies what install prefix to use. This is where the lib, include files are copied during the install process.
Once the make file is completed start the build and install process:
make make install
This does not run test cases but this is not possible on the host machine when cross compiling.
Step 4 – Integration
Integration into your project requires that you also link in several additional libraries that are required by protobuf.
Modify your projects .pro file and add the following line (including all dependant libs the protobuf requires):
LIBS += -L/PATH/TO/INSTALL/lib -linput_client -linput_utilities -lstdc++ -lprotobuf
Finally update QNX configuration with the path to the include directory.
Done.
Running Django tests in PyCharm for standalone apps
Over the last couple of years I have created several Django projects designed to be standalone apps. While it’s not to hard to create a custom test running to handle unit tests using:
django.core.management.call_command('test', APP_TO_TEST)
This technique unfortunately does not integrate very well into PyCharm’s built in testing tools (you also cannot debug).
You can however trick PyCharm into thinking it is running a Django project and everything works!
Create a manage.py file in the root of your project inside add the following:
# coding=utf-8 # Helper test runner for running tests for django standalone apps import os from optparse import OptionParser from django.conf import settings APP_TO_TEST = 'warthog' parser = OptionParser() parser.add_option('--DATABASE_ENGINE', dest='DATABASE_ENGINE', default='django.db.backends.sqlite3') parser.add_option('--DATABASE_NAME', dest='DATABASE_NAME', default='test.db')parser.add_option('--DATABASE_USER', dest='DATABASE_USER', default='')parser.add_option('--DATABASE_PASSWORD', dest='DATABASE_PASSWORD', default='') parser.add_option('--DATABASE_HOST', dest='DATABASE_HOST', default='') parser.add_option('--DATABASE_PORT', dest='DATABASE_PORT', default='') parser.add_option('--SITE_ID', dest='SITE_ID', type='int', default=1) options, args = parser.parse_args() settings.configure(**{ 'DATABASES': { 'default': { 'ENGINE': options.DATABASE_ENGINE, 'NAME': options.DATABASE_NAME, 'USER': options.DATABASE_USER, 'PASSWORD': options.DATABASE_PASSWORD, 'HOST': options.DATABASE_HOST, 'PORT': options.DATABASE_PORT, }, }, 'SITE_ID': options.SITE_ID, 'ROOT_URLCONF': '', 'TEMPLATE_LOADERS': ( 'django.template.loaders.filesystem.load_template_source', 'django.template.loaders.app_directories.load_template_source', ), 'TEMPLATE_DIRS': ( os.path.join(os.path.dirname(__file__), 'templates'), ), 'INSTALLED_APPS': ( "django.contrib.auth", "django.contrib.contenttypes", "django.contrib.sessions", "django.contrib.sites", APP_TO_TEST, ), })
Now enable Django support pointing at manage.py, create a Django Unit test and your good to go.
XCSS Parser
Working with CSS can be painful at times, and while there is the option of switching to SASS or similar I personally find these more even more annoying. Being able to quickly preview your work is useful.
CSS does actually have a few built in constructs to break up work better @import
does allow you to split up multiple files although this is not recommended for a production site as it increases the number HTTP requests.
My simple solution is a python app that will take your CSS files and combine them into a single file, at the same time removing any blank lines, this is not as advanced as a CSS minification tool but does a similar job. Combined with gzip compression on the server side for best results.
My biggest dislike with SASS and co is that the syntax is quite different from standard CSS, this means you cannot preview the output in your browser without a compile stage, slowing down the development workflow. My solution (thus far) does not introduce anything that is not supported already by your browser.
The only feature I added (which is entirely optional) is support for variables, again I wanted to ensure that the syntax is as similar as possible to standard CSS (they actually work more like a Make MACRO). A variable is defined using @define NAME VALUE;
and can then be referenced with $NAME
.
Anyway if your interested check out the project on BitBucket:
https://bitbucket.org/timsavage/xcss-parser/overview
Django Ajax form's updated
With the release of Django 1.2 the newforms module has had a re-factor, with validation pulled out into its own module (django.core.validation). Changes where required to the form introspection code that works out what validation is required client site. The upshot of these changes is that generating a JSON representation of the rules now requires a lot less code and is a lot simpler. =o)
The updated version code is now in the Django1.2 branch until I can work out how I will merge back into trunk while maintaining backward compatibility.
More to come soon…
Hacking up Gypsy
I recently came across a great little project (part of freedesktop.org) called Gypsy. Gypsy is a dbus enabled replacement for gpsd.
My own efforts are writing a GPS daemon are hosted on Google Code. My daemon was written entirely in python and similar to Gypsy published information via dbus. But Gypsy has a number of advantages:
- Support for the garmin protocol
- More complete implementation of NMEA and all DBUS information
- Designed to be efficient for both client and server
- Includes all configuration to run as a DBus service
My only concern was a feature that I had written into my daemon and this was the ability to record an NMEA stream and replay the stream at any time. This becomes very useful when testing client software.
To this end I have written two patches for Gypsy to firstly add support for TCP and secondly add support for writing a time delta to the NMEA logs output by Gypsy as well as providing a Python tool for replaying these logs via a TCP socket.
The patches have been submitted to the Gypsy dev’s or can be fetched from the Freedesktop bugzilla.
Update: after feedback from the gypsy maintainers I have updated my patch to simply record a delta to the log file. As of Gypsy 0.8 gypsy can now read from a named pipe the patch also includes a python script that can be used to replay the log.
More on IEnumerable and ICollection
Following up from my post yesterday on enhancing IEnumerable with additional features Phil Haack (Program Manager on ASP.NET for MS) posted a comment on the use of the Count Linq extension method and some of the dangers of using it with generic IEnumerables. With this in mind I would suggest updating my code to enhance ICollection which would remove this risk.
See Phil Haack’s post here.
Enhancing the ASP.NET MVC View fornext loop
A common problem with the default ASP.NET MVC view engine, when rendering collections in a for/next construct is the inability to easily determine where you are in the loop. For example a common way of displaying a menu is to use unordered lists and produce the following markup:
<ul class="menu> <li class="first"><a href="/home">Home</a></li> <li class="selected"><a href="/page-1">Page 1</a></li> <li><a href="/page-2">Page 2</a></li> <li class="last"><a href="/page-3">Page 3</a></li> </ul>
With the basic for/next loop over an ICollection producing this output is rather difficult.
My solution is an extension method for ICollection that provides additional information about the loop.
public static class CollectionExtensions { public static IEnumerable<LoopItem<TSource>> Loop<TSource>(this ICollection<TSource> self) { if (self == null) throw new ArgumentNullException("self"); var index = 0; var count = self.Count(); foreach (var item in self) { yield return new LoopItem<TSource>(item, index, count); index++; } } } public class LoopItem<TSource> { private int _count; internal LoopItem(TSource item, int index, int count) { this.Value = item; this.Index = index; this._count = count; } public TSource Value { get; private set; } public int Index { get; private set; } public bool IsOdd { get { return !this.IsEven; } } public bool IsEven { get { return this.Index % 2 == 0; } } public bool IsFirst { get { return this.Index == 0; } } public bool IsLast { get { return this.Index == (this._count - 1); } } }
With this code in place we can now do generate the menu from a collection with:
<ul> <% foreach(var item in menuList.Loop()) { var classes = ""; if (item.IsFirst) classes += "first "; if (item.IsLast) classes += "last ";%> <li class="<%: classes %>" > <%: Html.ActionLink(item.Value.Action, item.Value.Title) %> </li> <% } %> </ul>
This makes displaying lists of data with specific classes a lot easier and simplifies your views. This same piece of code can also be used for rendering tables with odd/even rows.
Enjoy!