Archive for April, 2008

Grappling with too many results

My fellow Nestorticulturalists,

good news, we recently rolled out some minor design tweaks in the hopes of making it a bit easier to sift through our database and find your dream home more quickly.

Specifically, we’re attempting to grapple with the situation when you search across large areas like London or Scotland where we typically have more than 50,000 homes. In these cases it’s difficult to argue that there is a relevant correct search result. While we do show a few houses or flats we focus on helping the user quickly narrow his search to a more manageable number of possibilities. We’ve tried to make this as clear as possible.

Have a look and please let us know what you think. Here’s a screenshot of properties in Wales:

Welsh regions

The problem isn’t just one we face in large geographies - we have it everywhere we have a density of listings - for example as seen here when you search for a home in East London

East London

As always, best of luck with your house hunt. Rest assured our work on usability is far from over, we have a few more design changes being tested right now on a subset of users. More soon.

Quantity and Quality

Fellow Nestoriticians!

Today we wanted to give you a bit more insight into some of the challenges we face in building Nestoria. In attempting to provide our users with the easiest way to search for property in the UK we consider four major factors: comprehensiveness, usability, relevancy, and freshness.

Comprehensiveness is seemingly the simplest to measure of these parameters. Essentially it is asking - “how many properties are there in the database?” As with many things though that at first glance seem simple, the actual answer is not so easy. The question is whether you measure the gross number of properties or the net.

Of all the raw properties that come in, we unfortunately find some that are spam and of course we don’t want to show those to our users. Next, we also attempt to remove non-residential properties. Then there is the significant number of sold or ’sold subject to contract’ homes that we need to strip out. Detecting all of these types of ‘bad’ listings is conceptually straight forward (which isn’t to say we’re perfect - don’t hesitate to let us know when one has slipped through our nets).

The final challenge we face is a bit more difficult. Because we have listings from many sources we often have to grapple with duplicates - when we have the same property from multiple sources. This is often not trivial because the same house can have a different description or slightly different address details. Often the data from different sources disagrees slightly; source A may tell us the property is a freehold, while source B thinks it’s a leasehold. With limited and/or conflicting information the decision about what is and what isn’t a duplicate isn’t always clear. And of course the universe of properties we have to consider is continually changing - homes are continually coming on and off the market.

One possible solution you might propose is to analyze the photos of the property. This occasionally works, but even if they are the same original photos they may have slightly different size, cropping, sharpness, red-eye-reduction (just kidding), or image quality. All which makes the images look the same for the human eye, but different for a computer. Here are some examples we found recently of duplicates with slightly different photos of the same house:

duplicates

Rest assured, dear property searcher, we’re continually fine tuning the Nestoria algorithms to catch them all and only show you relevant results, rather than showing you the same home again and again.

More about the other aspects of creating a compelling search engine experience soon.

Speaking at SOTM2008

Greetings Nestorinards,

Recently we announced our sponsorship of OpenStreetMap’s 2008 State of the Map conference in Ireland in July. Today I’m pleased to update that announcement with the addition that I will be one of the speakers at the event. I hope to see you there.

A provisional lineup has been announced and looks to be very interesting, and we’re very much looking forward to the conference. As part of the buildup to the event, the various speakers will be interviewed on the SOTM blog, and my interview went live yesterday.

For those who will be at the conference, please send any and all comments regarding the talk and any points you’d like me to cover in more depth. For those that unfortunately can’t make it to the conference we’ll of course be posting the slides here. And finally, for those of you desperate for more examples on the rhetorical skills of the Nestoria team here’s the overview of last year’s events where Nestoria team members have spoken.

In The Lab - Sponsoring Research

Nestorscientists,

After his appearance at AGI2007, Ed was approached by a professor from UCL about sponsoring summer research projects for students from their Master’s program in Geographic Information Science. I went along a few months ago to pitch our project ideas and I am happy to report that one student researcher has taken up our challenge. Christopher Osborne will be working with us this summer to find new and innovative ways to present property-related data to users on a map. Chris comes from a background in online mapping - having built map interfaces for Brent Council and other UCL researchers. He will work primarily on his own, but with ideas, guidance, and data sets provided by Nestoria. We’re pleased to be working with the UK academic community and look forward to seeing what Chris comes up with.

Nestoria Interview - Alex Singleton - UCL’s Centre for Advanced Spatial Analysis

A few months ago we featured London Profiler, a website created by researchers at University College London. For this month’s Nestoria interview we have the pleasure of talking with Dr Alex Singleton, a research fellow at UCL’s Department of Geography and Centre for Advanced Spatial Analysis about London Profiler and the various other projects the department does.

Alex, thanks for taking the time to speak with us.

1. What is the Centre for Advanced Spatial Analysis and what are the types of projects you work on?
The Centre for Advanced Spatial Analysis (CASA) is an initiative within University College London to develop emerging computer technologies in several disciplines which deal with geography, space, location, and the built environment. Researchers at CASA are involved in a broad range of research activities including developing models and simulations of urban areas, pedestrian movement, implementing 3D Globes, the geography and ethnicity of names and multiple projects around geographic Web 2.0 type technologies.

My A-Level geography teacher once described the discipline of Geography sitting at the “hub of the wheel of knowledge”, and although an undoubtedly biased view, this conceptualisation has flowed through into my personal research activities which tend to cross an interdisciplinary range of areas. However, broadly as a human geographer I am interested in how new data sources and technology can be used to help us visualise how the world looks, and from this information, how we can build better models of how the world works.

2. What’s new with London Profiler? What was the goal of the project and what sorts of feedback have you had from users?
The growth of Google Map type technology, and the ability to re-use these interfaces and data in third party systems has transformed Geographic Information Science; opening up the visualisation of how the world looks to a plethora of new users. However, despite this improved access to general map and satellite imagery, there are still lots of really interesting local information about the neighbourhoods in which we live which are not available through these new interfaces. This is really the crux of the research problem which I have been working on in the London Profiler project.

The London Profiler website is a repository of data about London Neighbourhoods, covering a huge variety of different domains such as health, crime, ethnicity, deprivation and education. The majority of the data we display are in the public domain, however, we have simply improved their presentation. Using technology developed in CASA we are able to transform any spatially referenced data into the tile based system used by google maps for their background (map or satellite) information. Our data is presented as another layer on top of the Google background maps, which we can fade in or out as required.

The website also provides the ability to add your own data on top of our maps from any publicly hosted KML file. For example, the KML feeds which Nestoria provide have enabled us to create a basic property search function on our site. This perfectly demonstrates how synergies between different data providers using web 2.0 methods of sharing and reusing information can create innovative new services. For example, if I were looking to buy a house; by adding the deprivation data on London Profiler and then searching for properties for sale, I could examine how the price of properties for sale might change depending on how deprived an area was. For people moving into London, who are not familiar with the local geography, this provides a very powerful tool.

We still consider the London Profiler site to be very much in beta, and in the future we have a range of plans including the addition of more data, testing alternate mapping technology and expanding the coverage’s outside of London - A national profiler would be very exciting!

3. What can sites like Nestoria do to better

  • cooperate with academic researchers?
    Supplying an open API is a great way to start building interest, however this is only useful to academics if they actually know about it. Therefore I think co-operation could be improved, firstly through promotional efforts to develop your brand awareness within academia about the types of services, but more specifically data, you can offer to academics for use in their research. This may involve coming along to some academic conferences, or perhaps delivering joint papers. Secondly, an important way in which we have conducted much of our previous knowledge transfer is through the joint funding of PhD students working on collaborative projects. These activities can take a variety of forms and offer a good foot in the door of many leading academic institutions.
  • make our data available for academic projects?
    My co-researchers and I have been impressed by the data we have accessed through the Nestoria API. This model of access is very different from how those nationally distributed data academics normally use are released, and in reality this places Nestoria well ahead of the game. It would be great if we could see census or some of our other prevalently used data disseminated using similar technology.

    I would not want to imply that there was anything wrong with the current Nestoria offering, it is very good. However, as a geographer I might look at how aggregations of the data you collect could be gathered across a number of spatial units (e.g. Wards, Output Areas), perhaps with the ability to query these information temporally. This could be really useful.

4. What are the challenges you think a vertical search engine for property like Nestoria faces?
It will be interesting to see how business such as this scale, as unlike those recently publicised web 2.0 successes (e.g. Facebook) where the value is linked to the size and interaction of membership (which could grow indefinitely), a vertical search engine’s value is based on the volume of relevant data you can collect, and the frequency of clicks through this information you can garner.

The first challenge with this model is that information collected is theoretically capped at a maximum as there could (in the case of Nestoria) be a situation where you capture all property for sale within a country, thus limiting any further growth through data capture alone.

A second challenge lies in how you get the information collected out to your potential users, so they can click through and generate your advertising income. This is slightly more challenging and requires a good understanding of your potential customer base. The provision of API and those other dissemination routes such as Google Widgets / Facebook integration are excellent examples of how this can be achieved.

Finally in terms of risk, it appears to me that the vertical search engine business models are perhaps too dependent on advertising revenue derived through click through “adverts”. I suspect that to mitigate some of this risk in the future it will involve the development of new income generation streams outside of offering consumers a free service. The data collected by Nestoria undoubtedly has great geographic value and there are multiple ways in which a range of corporate services could be created without undermining the ethos of the current Nestoria model.

Great answers Alex, thanks for offering us your insights. For those that haven’t yet seen it I strongly recommend checking out London Profiler, it is an amazing tool.

Alex is correct that there is a limit to the number of houses on the market, and in the UK and Spain in most regions we are well past that limit. A bigger technical challenge we now face is correctly identifying duplicates as we now very often have the same listings from multiple sources. We hope our recent release of embeddable historic house price charts and support for the hListing microformat are just two examples of us trying to release our data in a more usable format. This is an ongoing project and one we hope to have more to report on in the future.

We’ll keep refining the data quality and user experience. We look forward to seeing what new innovations academic groups like CASA present.

past Nestoria interviews: Tom Steinberg, Lelia Ferro, Lloyd Shepherd.

Nestoria tools in the wild

Greetings Nestorwegians!

Today we thought we’d feature some sites out there across ‘the internets’ using the Nestoria webmaster tools. For those that don’t know we offer a full array of widgets, property lists, dropin maps, co-branding tools, an even a full API to allow webmesters to offer their users relevant property information.

Here’s an example of our easy to set up co-branded property search in action on regional site this is Hartlepool.

nestoria cobranded search on thisishartlepool.co.uk

Moving on to a slightly more glamourous region (no offense intended to our much loved Hartlepudlian readers), we also recently came across our API being used on About Mayfair, a site about London’s poshest district (no offense intended to our occasionally beloved Chelsea readers).

So whether your site’s readers are hunting for a multi-million pound mansion on the edge of Hyde Park or a more affordable cottage by the sea we’ve got the tools to help you offer a compelling property search experience.

As always, we’re grateful for any feedback about our tools, especially our most recently launched historic house price widget.

Nestoria hListing support

Fellow Nestoris!

For some time now momentum on the internet has slowly been building for microformats and the semantic web.

More and more these data formats and technologies are moving from theory to reality, with recent adoption from internet giants like Yahoo! to innovative start ups like London-based Retangle (still invite only). The most recent beta version of popular internet browser Firefox supports microformats and may be a key step on the path to much wider adoption.

microformats

We here at Nestoria always have our eyes open for any possible new technology or technique that may help you find your next home more easily. Today we continue this tradition by rolling out support for the hListing microformat on our search result pages.

We should note, the format is still under proposal, and to our knowledge there’s only one known in-the-wild parser - so this is bleeding edge stuff that might be useful in the future rather than an amazing new functionality for property searchers today. Nevertheless, even the mightiest oak grows from the tiny acorn. No doubt future microformat application developers are grateful to find live examples. We look forward to playing our party in the future growth of the semantic web.

If interested, read about our other efforts to support open standards and the open source development community.