Show HN: 3D to 2D, point clouds flattening (geometry images)
github.comI have been trying for a while to put irregular things (point clouds) in regular grids, to use the grid itself as a data structure. Not because it is better, but just because grids are more structured than e.g. trees and thus easier integrate into ML pipelines. It all started here, with simple multi-indexing:
https://news.ycombinator.com/item?id=49042861
Then I realised that to make the multi-indexing procedure more effective, I need some kind of preprocessing, to make the points flat (2d insteed of 3d). When they are already flat I can sort them better.
Then I heard about mesh flattening, https://geometrycollective.github.io/boundary-first-flatteni... geometry images https://hhoppe.com/proj/gim/ and thought: this is something I need to try with point clouds, whenever they have a latent surfacic distribution. And well, it seems to work. At least now I can have bunnys dragon and elephants all to fit in the grid and not getting half of the legs, ears or whatever lost in the process.
Could someone explain to me what this is for and when it comes up?
Opposite of snark: active curiosity :)
It's realy the idea that Flat 3D object <-> Image, you can get a look at what the "founders" of geometry images said about that,
and how they apply it to simplify some mesh pipelines (hope it will feed your active curiosity !): https://hhoppe.com/proj/gim/
What I bring upon geometry images with my project, is the same idea for point clouds.
replace point[42] (index = 42) with point[row, col] (row, col of a learned matrix structure that store the points. Thus, the point cloud is stored as an image, and the pixels are the points).
Promising application is point cloud compression, but I can think of others: matrix storage is maybe not as accurate as e.g. kdtree storage, but very convenient to use, eg for neighbor querys, convolution...
Possibly UV mapping to draw a texture map?
Ah so as to "paint" an asset that's been scanned perhaps?
Could this let you conpress models into lossy systems like JpegXL?
Yes ! here I get x20 compression on the bunny for a very correct output. (pip install flatpoints[viz] pyjpegxl)
import flatpoints as fp
import pyjpegxl
import os
#============ Raw point cloud =============================
xyz = fp.load("bunny") #Required format: numpy array, (N, 3)
#============ Fit the flattener ===========================
single_face = True #Or false -> two images (left + right)
flat = fp.Flat(xyz, single_face = single_face)
#============ Build one (single_face) resp 2 images: ======
images = flat.to_image(xyz) #(1, n, n, 3) or (2, n, n, 3)
left_image = images[0] #(n, n, 3), right_image = images[1]
#/!\, normalisation, needed for pyjpegxl:
b = xyz.min(axis = 0)
a = xyz.max(axis = 0) - b
left_image_normalised = (left_image - b[None, None])/a[None, None]
#============ Compression, Decompression ==================
pyjpegxl.imwrite(f"compressed.jxl", left_image_normalised, quality = 98)
_, image_decompressed = pyjpegxl.imread(f"compressed.jxl")
xyz_decompressed = a*image_decompressed.reshape(-1, 3) + b #basically, that's it.
#============ Visual inspection ============================
_ = fp.Flat(xyz_decompressed, single_face = single_face).plot()
#============ Memory saving ============================
memory_before = xyz.nbytes
memory_after = os.path.getsize('compressed.jxl')
print(f"Initial memory: {memory_before / 1024:.2f} KB")
print(f"New memory: {memory_after / 1024:.2f} KB")
- doubt about perspective projection in your example
- where is x, y and z pointed towards and what is its range