Archive for the 'technical' Category

Summer of Coding Interns

This is the third summer of Lokku’s internship program, and we have been very lucky to bring in 3 talented engineers.  Yoav was the first to join.  He is working closely with MTM on a variety of projects related to listing quality and distribution.  Croydon Chris - just off his second year at Durham - arrived second.  He and Spiros are concentrating on further automating our ETL system.  Alistair came all the way from the University of Glasgow and started just yesterday.  His six month placement with Lokku is part of completing an undergraduate Masters Degree.  Alistair will be working with me on improving and scaling the system we use for listings de-duplication.  It’s always nice to have some new blood around to shake things up, and we hope to learn almost as much from our interns as they learn from us.

Nestoria Sponsors YAPC::Europe 2008

As mentioned before, the Nestoria engineering team will be attending YAPC::Europe 2008 where MTM and myself will be speaking.  We are also very pleased to now be an official sponsor of the conference.  As voracious consumers of free and open source software, it’s good to have a chance to give a little something back and to actively participate in the lively perl community.  Thanks to the Copenhagen Perlmongers for organizing what looks to be another great YAPC.

Nestoria Engineers at YAPC Europe 2008

Hello Nestorcandinavians,

MTM and I just got word that our talk proposals have been accepted for YAPC Europe 2008. We had a good time at last year’s event and are proud to be actively contributing this time around.  I am going to speak about the various Perl profiling modules and MTM will relate some of the lessons learned building a high volume/low maintenance ETL system.  Alex and Spiros will also be present in Copenhagen, so the entire team will be taking a break from building new features and concentrating on learning a few new tricks.  We look forward to seeing friend old and new in August.

In The Lab - Sponsoring Research

Nestorscientists,

After his appearance at AGI2007, Ed was approached by a professor from UCL about sponsoring summer research projects for students from their Master’s program in Geographic Information Science. I went along a few months ago to pitch our project ideas and I am happy to report that one student researcher has taken up our challenge. Christopher Osborne will be working with us this summer to find new and innovative ways to present property-related data to users on a map. Chris comes from a background in online mapping - having built map interfaces for Brent Council and other UCL researchers. He will work primarily on his own, but with ideas, guidance, and data sets provided by Nestoria. We’re pleased to be working with the UK academic community and look forward to seeing what Chris comes up with.

Nestoria hListing support

Fellow Nestoris!

For some time now momentum on the internet has slowly been building for microformats and the semantic web.

More and more these data formats and technologies are moving from theory to reality, with recent adoption from internet giants like Yahoo! to innovative start ups like London-based Retangle (still invite only). The most recent beta version of popular internet browser Firefox supports microformats and may be a key step on the path to much wider adoption.

microformats

We here at Nestoria always have our eyes open for any possible new technology or technique that may help you find your next home more easily. Today we continue this tradition by rolling out support for the hListing microformat on our search result pages.

We should note, the format is still under proposal, and to our knowledge there’s only one known in-the-wild parser - so this is bleeding edge stuff that might be useful in the future rather than an amazing new functionality for property searchers today. Nevertheless, even the mightiest oak grows from the tiny acorn. No doubt future microformat application developers are grateful to find live examples. We look forward to playing our party in the future growth of the semantic web.

If interested, read about our other efforts to support open standards and the open source development community.

Durham Mapping Party

Hail Nestorlinians!

As you may recall, a few months back we announced our sponsorship of Living with Dragons. Gregory Marler, the man behind the project, is attempting to live without copyrighted maps. An interesting, and entertaining, experiment. He’s slowly but surely mapping Durham using OpenStreetMap; you can follow his progress. To speed things up, a Durham Mapping Party is being organised for the 7th and 8th of June, and we hope many a loyal Nestoria fan will participate.

You can check the progress of the OSM volunteers in Durham and elsewhere in the UK via the experimental OpenStreetMap version of Nestoria. Compare our mainsite version of Durham and the OpenStreetMap version. As you can see there’s still work to be done in Durham. Other recent mapping parties have proven very successful and more and more cities are being mapped in great detail, and I’m sure it will be the same in Durham. Other upcoming mapping events in the UK include Birmingham, Southwest Surrey, and Wales.

Best wishes to all the mappers out there!

Tech events - Oxford Geeks, London DBAs

Mike and I are going to meet fellow MySQL users this Thursday at the London MySQL Meetup. You need to login to see all details and add yourself to the list, but here are the basics:

Thursday, January 24, 2008 at 6:30 PM
The Lamb
Lambs Conduit Street, Bloombury WC1
[info] [map]

This is Lokku’s first time at a MySQL event. Having just finished the MySQL High Availability training this week (one hour ago, I am still sitting in the training center), I’m sure to have lots to talk about.

In other news - I am going to give a microslot talk about Mapstraction at the next Oxford Geek Night:

Oxford Geek Night
Jericho Tavern
56 Walton Street, Oxford

Mapstraction is a Javascript library that we used to build the OpenStreetMap-based version of Nestoria.

I hope I meet some of you at one or both events!

Browser Wars at Nestoria (Take 2)

  • You are important to us (Nestoria reader/user)
  • You use a browser to get to us
  • Therefore, the browser you use is important to us too

3 simple truths, just as true now as they were almost a year ago, when we brought you our original ‘Browser Wars’ post.

With several recent developments in the area (the sad demise of Netscape, and the rise of questionable ‘viral’ campaigns), we thought now would be a good time to provide you with the 2007 data for Nestoria browser usage. Please click to enlarge the graphs.

Browsers 2007

No major surprises overall. The Windows-bundled Internet Explorer led the pack, with a 2007 average of 84% of all visits. Firefox came in a respectable second, with 12% of visits. Safari representation was uncharacteristically high at 4%, Opera bringing up the rear of the big hitters with 1%. Interestingly, our statistics vary considerably from W3School’s, which has Firefox only 20% behind IE.

IE?Ĵs ShareFirefox?Ĵs ShareIE vs FF

So what about specific IE / Firefox trends? Well, there’s possibly a very slight negative trend in % of IE use, and a very slight positive trend in % of Firefox usage. Relative share seems fairly stable.

IE Version uptakeFirefox version uptake

Possibly the most interesting data is that for version share. IE7 overtook IE6 around October 2007 - showing that Microsoft’s big push hasn’t been in vain. However, there’s still a high number of IE6 users - I would hazard a guess that these are mostly within the workplace, where managers have not yet made the move to IE7. Unhindered by similar issues, the switchover from Firefox 1.5.x to 2.x was quick and progressed almost to 100% usage of 2.x.

To conclude
What do these statistics mean? For us, they mean time spent testing and tweaking our site on the appropriate range of browsers. For you, faithful user*, this hopefully translates into increased site performance and Property Hunting Pleasure?Ѣ?Ć.

As always, please feel free to fire off a comment, or send us some feedback.

* For the crazy cats accessing Nestoria on their PSPs, PS3s, Wiis, iPhones, we’ll do our best.

?Ć This isn’t really trademarked

OpenStreetMap version of Nestoria

Greetings fellow Nestorhamians!

One of the things users like most about the property search results on Nestoria is the way we use maps. So it’s only natural that since day one we’ve kept a close watch on innovation in the online cartography space. We regularly interview thought leaders and innovators, we’ve sponsored projects like Mapstraction and Living with Dragons, and spoken at the odd industry event. Along the way we were very proud to be cited by Google as the Google maps case study.

Today we’re happy to play a small part in another milestone on the long path of neogeography innovation. Like many folks in the geocommunity we’ve followed OpenStreetMap with interest. Started in 2004, OSM is a volunteer project to map the world. At first glance that might sound absurd, but keep in mind that Wikipedia sounded silly at first as well. Today Wikipedia is used by literally millions of people every day. Similarly, most of the software used to power the internet was created by volunteer driven projects.

Nevertheless, many people are skeptical that this sort of community driven data collection can work, that it can actually create usable data.

Fair enough, seeing is perhaps a steps towards believing. We’ve created a parallel version of our site that uses maps from OpenStreetMap maps rather than the traditional Google maps. Every page on our site can also be viewed in OSM mode, so folks can do a head to head comparison.

Here’s an example, a screenshot of Chelsea with an OpenStreetMap map:

OSM Chelsea

and here’s a screenshot of Chelsea with Google maps:

OSM Chelsea

As you can see the OSM maps in many part of the UK are very good. Have a look for yourself at openstreetmap.nestoria.co.uk, or by simply changing the www in any Nestoria search results page URL to openstreetmap.

OpenStreetMap seems to really be gaining momentum. If you aren’t familiar with the project, please check it out. Recent highlights include the donation of some large geodatasets by commercial organisations and municipalities that realize the community is better placed to maintain the data, and the comprehensive mapping of several large cities. OSM is now used by several universities, including Oxford.

One of the key advantages of OSM is that it’s volunteer driven, meaning maps can be made of anywhere people feel like making maps of. There are many parts of the world professional geodata companies don’t focus on, for understandable financial reasons. As examples, OSM volunteers have recently mapped Stanley in the Falkland Islands, cities in the Philippines, and will be carving up India in February.

One question this experiment will no doubt raise is whether we plan to move away from Google maps. No, we don’t. Google maps are excellent. It’s been especially impressive to see the functionality continue to evolve (and push others in the industry to improve) over the last two years. As believers in the open source movement we wanted simply wanted to create a way to highlight some the amazing progress of the volunteers behind OpenStreetMap.

Some technical details for those interested in how it all works. To be honest it was pretty simple thanks to Mapstraction, and the amazing flexibility of the Google Maps API. Anyone interested in embeddable maps, or even javascript in general should check out Mapstraction. Andrew Turner recently wrote a great tutorial; a good place to start. Also perhaps interesting is our interview with Steve Coast, founder of OpenStreetMap, from last year.

On a final note, this experiment isn’t just limited to the UK, we’ve also created the same OpenStreetMap version of Nestoria in Spain. Just surf to openstreetmap.nestoria.es and let the fun begin. In general Spain is less well mapped, though there are some areas like Sol in central Madrid that are very comprehensive.

Enjoy, and, as with everything we do here at Nestoria, please let us know what you think.

Nestoria Rank update - December 2007

Fellow Nestorwegians,

The time between Christmas and the New Year is a natural chance to look back on the last year and what we’ve accomplished. The next few days we’ll do a few more reflective posts summarising some of the accomplishment of the past year.

First up we thought we’d revisit Nestoria Rank, the algorithm we use to try to present relevant results to property searchers. Over the last few months we’ve continually experimented with changes designed to get people to the right house or flat more quickly.

Some of these changes have been very visible, others are more related to the algorithm itself. Let’s go through some of the more visible improvements first:

We now allow you to search by property characteristics.

screenshot of searching for property characteristics

This means searches like: an unfurnished two bed flat to rent near Little Venice with a balcony are now easily possible (right now we have 111 of them in the database)

Likewise, we’ve added the ability to filter your search only for “new builds”

We know newness to the market, or ‘freshness’ as we call it is a critical part of how people search for their next home so we now show you how long a property has been on the market and allow you to sort properties by age.

We’ve also integrated more and more local content to provide context to your property search. Most recently we’ve updated the London tube lines to add the London Overground, but we also added sports facilities and parking spaces in the last few months.

But those are just some of the new features you as a user can see. Rest assured we’ve been equally hard at work optimising the backend systems that you don’t see. Over the last few months we’ve been experimenting with systems that learn from individual user behaviour and adapt in real time. This is still in test mode, but we’re almost there. So don’t be surprised if the guy sitting next to you in the office starts getting Nestoria different search results than you.

Our thanks go out to everyone for the feedback we get on new features, but especially we’d like to thank those folks who volunteered to be betatesters.

So, what does 2008 hold? Well in the immediate future you can expect better integration of the metadata we recently made available via our API, and we’re always playing with data visualisation.

BTW - If you don’t like the way we’re displaying search results you have two choices: please let us know what we could do better and we’ll try to run a test with your suggested modification, or you can try your hand at our API and build your own interface. In the last month there have been many interesting uses of the API.