...AI is currently the subject of great enthusiasm. If that enthusiasm doesn’t produce a bubble conforming to the historical pattern, that will be a first.
> Most of the Group 3 dogs—which had previously learned that nothing they did had any effect on shocks—simply lay down passively and whined when they were shocked
What a cruel time for experimenting on animals the 1960s were...
Animal cruelty is alive and well in the factory farming industry, at a yearly scale orders of magnitude higher than the sum of all research experimentation in science during the 1960s.
It all kind of depends on each other. More light, means longer recycle times on the speedlights or higher iso, more noise. Longer exposure isn't an option with speedlights, using continuous also has it's downsides, things may start to shake..
The bumblebee was my first attempt, the tracking didn't quite work, so you get ghosting. Others too have ghosting, usually happens when part of the insect moves, while shooting (which takes 4h). They dry and crumble after a while.
Thanks for the links, that is great to know.
I'm not quite sold if it's the better approach. You'd need to do SfM (tracking) on the out of focus images, which with macro subject can be really blurry, I don't know how well that works.. and a lot more of images too. You'd have group them somehow or preprocess.. then you're back to focus stacking first :-)
The linked paper describes a pipeline that starts with “point cloud from SfM” so they’re assuming away this problem at the moment.
Is it possible to handle SfM out of band? For example, by precisely measuring the location and orientation of the camera?
The paper’s pipeline includes a stage that identifies the in-focus area of an image. Perhaps you could use that to partition the input images. Exclusively use the in-focus areas for SfM, perhaps supplemented by out of band POV information, then leverage the whole image for training the splat.
Overall this seems like a slow journey to building end-to-end model pipelines. We’ve seen that in a few other domains, such as translation. It’s interesting to see when specialized algorithms are appropriate and when a unified neural pipeline works better. I think the main determinant is how much benefit there is to sharing information between stages.
You can definitely feed camera intrinsic (lens, sensor size..) and extrinsic (position, rotation..) into the SfM. While the intrinsic are very useful the extrinsic not actually that much. In no way can you measure the rotation good enough, to get subpixel accuracy. The position can be useful as an initial guess, but I found it more hassle than worth it. If the images track well, have enough overlap, you can get exact tracking out of them without dealing with extrinsic. If they don't track well, extrinsic won't save you. That was at least my experience.
Nothing fancy. Postshot does need a nvidia card though, I have a 3060Ti. A single insect, with around 5 million splats takes about 3 hours to train in high quality.
