I'm the first author of this, happy to answer any questions. It's great to see this on HN.
PS. I'll be graduating this July with my masters.
As far as I can tell it's not directly related to yours - but are you aware of any software that uses say, Android OS and a camera or pre-existing photos of the inside of a house/office/building to create a 3d map? Matterport seems like it might but their Android app is not going so well.
Ben Claremont has done a lot of reviews of 3D virtual tour software, see https://youtu.be/uKkQQ0aHRSc
Ben’s reviews are the best I’ve seen, but he is far from the only reviewer in this space.
Unfortunately, most of the software in this space does not actually generate a floor plan, even if they do create a 3D model of the space (like Matterport). There is one program he highlighted which is bundled with a service from the company that you can use to process your pictures and generate a floor plan among other things, but some of that work is still done by human beings.
I’m still looking for a good consumer grade program that can do photogrammetry of a space and turn that into a floor plan. I’ve seen attempts to do this with phones and cameras, and the LIDAR scanner that Apple has recently introduced has improved the situation, but IMO it is still not there yet.
But please correct me if I’m wrong. I would love to be wrong.
Did a bit of googling and this came up: https://canvas.io/. But, it's not for Android.
Ahh, yes — they used to support the Structure.io scanner that you could strap to an iPad or iPhone. I bought one of those. I could never get it to work.
Edit: the Canvas.io software really wants the latest iPhone or iPad devices with LIDAR scanners built-in. It will supposedly run on lesser hardware, but with greater error in the constructed 3D map. Fine for a sample to get an idea of whether or not it might work for you, but not really enough to actually use for anything useful.
Congratulations! Always great to see SFU students in the wild :)
You know it would also be useful to work in reverse. I get frusterated when house listings don’t include a floor plan. How cool if a program could go through the photos and guess a floor plan. Obviously a hard problem.
A hard problem, but back in 2008, Microsoft Photosynth could stitch together a user’s collection of images into a 3-D model. I remember seeing a detailed model of the exterior of the Notre Dame cathedral.
For people who have physical access to a property, I wonder if you could use a variant of SLAM like the robot vacs - maybe hack one of those laser tape measures to plug into your phone as you walk around the house.
Photosynth was a great app, really disappointed when it was discontinued. Does anyone have the backstory on that? It seemed to have a leg up on Google’s street view/cardboard app at the time.
Bringing a laser tape measure and a pen to a physical viewing, and just doing this manually, gets you 98% of the way.
It's not automatic of course, but you can do it pretty discretely and cover a whole house with measurements in ~ 10 minutes. I did it every single time when we were looking at houses.
Also always brought a powerful flashlight and a small spirit level.
Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.
There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.
Estimating the global camera pose using non-overlapping photos is a challenging problem. This is doable if the photos densely cover the house (e.g. video frames).
However, if the global camera poses are available, we can detect surfaces in photos and project them reasonably well into 3D. In the past, I have used this work to achieve something similar: https://github.com/NVlabs/planercnn
In addition to Matterport, mentioned in another comment, there's FloPlan, which has the agent walk around the house taking a video of the baseboards, and constructs a floorplan from that. Not entirely accurate, but good enough for a rough idea of the layout of the space.
I believe it can also use the LiDAR sensor in newer phones, but I haven't tested that.
An easier sell than Matterport, which (IIRC) is a $5K camera. FloPlan just charges a per-plan price (again, IIRC).
Floplan is based on the technology from Cubi Casa at https://www.cubi.casa/ but FloPlan is only available for MLS property listings.
If you want the generic service, you have to go to Cubi Casa.
Matterport will also work with a phone, so the fancy camera is optional.
I don't know if this is available where you live, but where I live the county assessor maintains a web application that contains information about each parcel and the buildings theron, including simple floorplans.
In Austin, Texas, you can get a “platte map” which will show the boundaries of the property and a simple outline of where the known structures fit in that map, but that doesn’t help with indoor floor plans.
Very interesting. We actually are looking at viable approaches to get usable indoor maps for large buildings currently. We are building map based productivity software for workers in factories and offices. Maps are important for our use case but not the core thing we do. We actually partner with other companies for maps and positioning typically.
Indoor mapping is a hard problem because typical map providers tend to ignore private spaces, which is where maps are most needed. E.g. Apple, Google, Here, etc have great maps for a very limited set of public buildings but they don't really have solutions for indoors.
We know of a few companies specializing in this but it typically involves a lot of manual work and tends to have a high cost, require a lot of hand holding, and some very niche mapping skills. Also, a lot of indoor maps are intended for e.g. architects, contain a lot of clutter, or are generally not well suited for use on e.g. mobile screens.
It's an interesting problem to start tackling because we are getting kind of a critical mass of technology that will be able to start delivering pretty accurate positioning to commodity phones. There are all sorts of things happening in the industry related to that. It's a bit similar to when GPS became common on phones and it became a mass market in a few short years. Before that you had all sorts of niche stuff. After that you had ridiculous amounts of user generated content, open street maps, foursquare, etc.
Hi! If you're looking for a developer friendly indoor mapping suite (both for editing and viewing/navigating), check us out - www.mappedin.com
Totally agree that it's an overlooked problem and we've been quietly working to solve it for years. Early on we realized the key was to build "everyman" mapping tools that facility managers can use themselves to keep data up to date.
We're increasingly focused on our developer-facing tools and I'd love any feedback if you end up taking a look! (https://www.mappedin.com/mapping/sdks/)
BTW the demo link in the sample app does not work:
We are in contact with mappedin but thanks for the tip!
Hey, producing indoor maps of private spaces is an area I've been working in for the past few years and I know exactly what you're dealing with.
I'd be happy to talk more about this if you're interested - email is in my bio.
Sounds like your company and the company I work for operate in similar areas. We focus on the indoor routing and positioning aspects, but are quite constrained by the effort of converting a floor plan or PDF of a CAD into a map. May be worth us connecting on LinkedIn to see if our companies could mutually benefit each other. My profile is my username here, without the “i am”.
This is neat, do you have any future ideas of making this into a product?
This is definitely a thought I have considered. It could potentially ease lot of hassle related to 3D scanning of houses by real-estate companies. Very often, those scans require expensive equipment, professional expertise to use those equipment and considerable time and effort to capture the entire residence. Even then,you will find many holes in those scans and floating debris. With our work, one only needs a sketch of the floorplan and photos taken from that house. Plan2Scene will use data to infer appearance of unseen surfaces.
For instance, look at this sample 3d scanned house: https://matterport.com/en-gb/media/2486. Make sure to switch to the doll-house view from the toolbar. There are many holes in the scan.
hi yasith, fancy running into you here :p
This seems to be really well done.
Rent3D++ dataset is not available for download, there is only a Google form they want you to fill out and no automatic email is sent upon submission. It's great that the source code is available, but it kind of defeats the purpose if the data it relies is behind a wall or at someone's whim before you can access it.
Hi, This is because the Rent3D dataset we extend has a "non-commercial use only" term.
How do we get access to it? I filled out the form and haven't heard anything back.
I did got a delivery failure on one of the emails I sent out. Can you submit again just to be sure?
Here is the form: https://docs.google.com/forms/d/e/1FAIpQLSfl4muDFf0qktqtWhj6...
Sorry for the trouble.
Get a daily email with the the top stories from Hacker News. No spam, unsubscribe at any time.