Neural networks for A:M?

[robcat2075]

Neural Networks learn to solve problems by being given examples of the problem solved.

 

Could a Neural network learn to convert polygon models to splines if it was given enough examples?

 

How would we get started on doing this?

[fae_alba]

Shudder...That's some heavy math lifting! Seriously though, I can see where that would be useful. Or how about fixing issues with a mesh, improving a walk cycle, lighting. Just about any decision making process could be automated to some degree with a network.

[robcat2075]

I'm reading that there are open source collections now that contain the essential mechanisms. What's left is to figure out is the input and out put.

 

Maybe text files of the input polygon and output spline models would be enough to get it going if you had enough examples. The amount of examples it needed might be huge however.

 

It might not even matter whether the computer understands the files as 3D objects.

[Rodney]

This sounds slightly familiar so I think some folks may already be on board to some extent... optimizing to quads that is... not necessarily splines.

You could probably look at Open Subdivision as an inbetweening point between Polygons and Splines.

Because the industry is likely interested up to the point of Subdiv surfaces the new focus might be on the latter bridge between Subdiv surfaces and Splines which might not get as much attention.

 

The network I'm thinking of has something to do with a master database of all possible encounters and then the AI tries to select the best outcome.

Because the quad 'patches' are fairly interchangeable with quads of other varieties It's the poles (extraordinary vertices) that would need to be assessed and resolved.

[nemyax]

Could a Neural network learn to convert polygon models to splines if it was given enough examples?

 

This problem is purely deterministic and can be reliably solved algorithmically. I don't think you'd gain much by training a NN to solve it. I rather think you'd only lose in this particular case.

[robcat2075]

But it's never been solved algorithmically. There is substantial judgement about where to put splines that is learned from experience and from seeing examples.

[nemyax]

I'm guessing you mean retopology of dense arbitrary-topology geometry, not something as simple as converting poly edges to splines. There are multiple methods that solve the generic problem of retopology, only they produce polygonal output. It's just that nobody has cared enough about spline patches to adapt the techniques to them.

[Rodney]

It's just that nobody has cared enough about spline patches to adapt the techniques to them

 

And that would be the drive behind those of us who do care about about adapting them (but lack sufficient means to get the job done).

Everyone has to start somewhere.

 

I will admit, the insufficiency of interest is considerable.

Pressing ahead is a bit like the process of animation itself where given two keys a (potential) breakdown is formulated and then the inbetweens are mathematically formulated.

That breakdown is the element that requires decision making and thus would be a candidate for AI screening and testing.

 

With regard to polygon conversion, an issue with that decision making process is that it isn't a yes/no proposition on where to emphasize continuity except in the case of two choices being presented (presumably among many). Thus this would appear to be a matter of testing one hypothesis and running it to its logical conclusion and then starting afresh and running through the process again. Some measures of success would need to be identified at the outset and those would be refined in accordance with (presumably at least initially) some form of human interaction that determines if an outcome is deemed successful. Success I would think would be a probability (percentage) of historical success.

 

This might not be as difficult as some think or it might be much harder.

Imagine that the system is constrained to only take 'right turns' when it comes to a crossroads or to only proceed forward and never less than at a specific view of what is considered to be a head. What about weight? Could the objects mass (or various areas of mass) be used to drive continuity? Most of this is likely already investigated (and implemented) in methods converting polygons to subdivision surfaces.

 

Added: A:M uses a "is flat?" test in aspects of its programming and an "is patch?" test might be similar. On the surface (pun intended) this would be a boolean operation of true/false but a score given would suggest to what degree that test was/is currently regarded as true. Over time the better patches would theoretically be favored and in the case of a tie the user might be able to shift through the available options on the fly (or depending upon what was within a renderable view (this is similar to other A:M philosophy of scale where an artifact (read: error) is still present but simply scaled out of view. In this sense a dynamic mesh might be one where discontinuities always faced away from the (camera) view, collecting themselves on the back side of the mesh. Immersive approaches such as gaming where all sides of a model can be seen at the same time would not be an issue because each 'camera' renders its own view and thusly presses the discontinuities out of view. The cameras simply pass control on to another/appropriate camera to complete the illusion. Granted, this isn't solving the problem but dealing with it by prioritizing errors, solving known issues and categorizing and sorting those that remain for subsequent resolution.

 

 

 

 

.

[Rodney]

A question (and potential answer) occurs to me regarding the question of...

 

Why would/should we tessellate a 3D objects surface?

 

The obvious answer might be so that the object can be presented on hardware optimized for that tessellation.

The answer that suggests itself to me would be to normalize the size and shape of every surface face/facet on a mesh.

This tessellation product would be presented to the viewer immediately (pending refinement).

The secondary use of the tessellated product would be to inform the creation of quad surfaces.

That information would be compared to the weighted (interim) surface as identified by continuity in the original (or a chosen proxy) mesh.

This continuity would dictate priority.

Areas of discontinuity would be set aside for later processing.

Areas of continuity would be solved (high risk areas would be shifted aside for later assessment and categorization).

Because the tris can be formulated and presented on the fly they need only be retained in short term memory and discarded when no longer required (i.e replaced with higher fidelity surfaces).

 

Bottom line: Tris may be sub-optimal in the greater scheme of things but they can still be exploited.

This is yet another reason to lean into optimized conversion of polygons and splines.

Converting splines to polygons appears to be already solved.

[robcat2075]

Using Polygon geometry to derive splines is probably a mis-step.

 

When we model in splines we don't first imagine of the object in polygons and then convert them to splines. We look at the overall shape and contours and decide where splines are needed to mimic that shape.

 

That is what I would like a neural network AI electronic brain to learn to do.

 

You'd probably still need to give it an OBJ as the starting point, to leapfrog past the weakness of AI's visual pattern recognition skills.

[Rodney]

RE:

Using Polygon geometry to derive splines is probably a mis-step.

 

 

and

You'd probably still need to give it an OBJ as the starting point

 

 

I"m not sure I follow you as those two statements would seem to cancel themselves out.

Although OBJ models that do not consist of tris might already be considered pre-optimized to the extent they have established acceptable routes of continuity.

There's a huge gap between polygons and splines that has to be defined somehow.

 

But... yeah... go AI!

[Rodney]

I've skimmed a few documents looking for the ones I was referring to before that sounded familiar but didn't find that reference.

I did keep running across this document from 2012 that is related:

 

http://www.meyumer.com/pdfs/yumer-kara-JCAD2012.pdf

 

 

 

Aside unrelated to this document: In thinking about processes that a AI nuerual network might consider certain efficiencies readily present themselves. For instance, if the input model is symmetrical then only half of the mesh needs to be solved. So one test might be a 'is symmetrical?' testing branch ran early in the process to separate the wheat from the chaff. Considering further this alone might be a useful consideration in conversion even without AI where benefits can be gained by using what to the end state desired might initially be deemed sub-optimal models (i.e. those with large regions of symmetry). The modification to achieve asymmetric is then of necessity left for after the conversion.

 

So thus far I see these potential tests:

 

Is flat? The greater the incidence of flatness the easier the surface can be resolved into quads.

If extraordinarily flat (i.e. a plane) then consider position relative to camera and store data.

 

Is uniform in size (or vertex distance)? The easier to use one calculation to resolve while maintaining at same relative size.

If non-uniform in size or distance use optimization to move points until uniformity is achieved.

Initial test would be to draw a baseline. Subsequent tests would consider the distances with respect to irregularities (isolated islands).

 

Is symmetrical? A set of tests determine repetition/mirroring with goal of reducing assessments and calculations while determining the presence of 'isolated islands' (abnormal regions/extraordinary vertex).

If symmetry is found store as clone/copy of the first exemplar.

 

Is extra ordinary vertex? Isolate and categorize pending importance and complexity.

These areas of the mesh may require user input to resolve. Ideally the user could toggle through the series of (working) solutions and select what is most desirable. This search/selection might be accomplished thrugh drawing on the screen and the database would locate a solution that resembled that line.

 

Now returning to the question:

 

How would we get started on doing this?

 

 

The first step is usually to inventory what is already on hand and then identify shortfalls/requirements.

 

What neural networks are available?

[robcat2075]

I've skimmed a few documents looking for the ones I was referring to before that sounded familiar but didn't find that reference.

I did keep running across this document from 2012 that is related:

 

http://www.meyumer.com/pdfs/yumer-kara-JCAD2012.pdf

 

 

 

They're on the right track. It shows the neural network can deduce a shape. Not much spline continuity there except for the most basic surfaces.

[nemyax]

RE:

Using Polygon geometry to derive splines is probably a mis-step.

 

 

and

You'd probably still need to give it an OBJ as the starting point

 

 

I"m not sure I follow you as those two statements would seem to cancel themselves out.

You need a representation of the surface you want to approximate, and OBJ is as good a representation as any.

[robcat2075]

RE:

Using Polygon geometry to derive splines is probably a mis-step.

 

 

and

You'd probably still need to give it an OBJ as the starting point

 

 

I"m not sure I follow you as those two statements would seem to cancel themselves out.

 

I figure you need the point cloud and the surface it suggests, but the facet edges between the points don't necessarily have a desirable spline path among them