Ogre shared bones with two bone-driven methods

foreword

Recently, Ogre is used in the business to drive virtual character bones based on 3D key points, but two problems are encountered:

  • The bones of the mesh such as the body, head, eyes, clothes, etc. are separated, but the bone structure is the same, and the shared bones need to be set
  • When driving, you can directly modify the bone rotation amount, or store the rotation amount in the animation frame, which will automatically insert frames according to the playback time interval

International practice, reference blog:

Ogre3D realizes character dressing

[Ogre-windows] Rotation matrix and position analysis

Ogre Dressing System shareSkeletonInstanceWith

Code

The following respectively include: shared bones, joint drive, animation frame drive, pit encountered

The motion created by the joint driving and animation frame driving methods is straightening and bending of the left calf, straightening and bending, back to straightening and bending, and so on.

shared bones

The core function is shareSkeletonInstanceWith, which can specify whose bones are shared with whom

However, it should be noted that the shared and shared bones have the same topology, otherwise an error will be reported.

If you want to force sharing, you need to use _notifySkeleton, the official description is as follows:

Internal notification, used to tell the Mesh which Skeleton to use without loading it. 
@remarks
This is only here for unusual situation where you want to manually set up a Skeleton. Best to let OGRE deal with this, don't call it yourself unless you really know what you're doing.

That is to say, tell a mesh to use another bone, but don't use it lightly, because it is easy to cause problems, and you will find out later in the experiment.

The extra code will not be posted, just read the source code at the end of the article.

First read three models: two Sinbad.mesh and one jaiqua.mesh

//main model
ent = scnMgr->createEntity("Sinbad.mesh");
SceneNode* node = scnMgr->getRootSceneNode()->createChildSceneNode();
node->attachObject(ent);
// Vice model 1
ent1 = scnMgr->createEntity("jaiqua.mesh");
SceneNode* node1 = node->createChildSceneNode();
node1->setPosition(10, 0, 0);
node1->attachObject(ent1);

//Vice model 2
ent2 = scnMgr->createEntity("Sinbad.mesh");
SceneNode* node2 = node->createChildSceneNode();
node2->setPosition(-10, 0, 0);
node2->attachObject(ent2);

Then share the bones:

ent1->shareSkeletonInstanceWith(ent);
ent2->shareSkeletonInstanceWith(ent);

An error will be found:

case Exception::ERR_RT_ASSERTION_FAILED:    throw RuntimeAssertionException(number, desc, src, file, line);

It is because the bones of jaiqua.mesh and Sinbad.mesh are different, so for jaiqua.mesh must be added:

ent1->getMesh()->_notifySkeleton(const_cast<SkeletonPtr&>(ent->getMesh()->getSkeleton()) );

This will run successfully, as shown in the figure below, from left to right are: secondary model 2, main model, and secondary model 1; since the secondary model 1 and the main model have different bones, they cannot be driven normally.

The role of shared bones is that sometimes the same model is divided into several parts for design. For example, the head and body are separated, which is convenient to separate the expression drive and the limb drive, but they are both complete human skeletons when they are designed. So you need to share the bones to do a synchronization.

Modify the drive for joint rotation

Animation Frame Drive Method

There are two types, one is to create and play while the other is to create and then play

Create first and then play

First of all, you need to know the animation duration, frame rate, and playback speed you want to create. In order to test the interpolation effect of frames, I created a 6s animation frame sequence. First initialize:

anim = skel->createAnimation("myanim", 6);
anim->setInterpolationMode(Animation::IM_SPLINE);
tracksnew = anim->createNodeTrack(lknee->getHandle(), lknee);
createAnim(); //Create animation frames

//animation play
as = ent->getAnimationState("myanim");
as->setEnabled(true);
as->setLoop(false);

The next step is to create animation frames, the specific creation method, in previous blog It has been introduced, and the code is directly pasted here:

void MyTestApp::createAnim() {
	for (int i = 0; i < 6; i++) {
		TransformKeyFrame *newKF = tracksnew->createNodeKeyFrame(i);
		Quaternion quat;
		quat.FromAngleAxis(Degree(i%2? 0.0f: -90.0f), Vector3::UNIT_X);
		newKF->setRotation( quat);
		prev_rotate = quat;
	}
	ent->refreshAvailableAnimationState();
}

Note that after the creation, you need to refresh the state of the animation, otherwise the modification will not take effect.

Finally, set the playback interval in frameRenderingQueued:

as->addTime(0.033333);

Indicates that the next 0.0333 frames of data are played each time, if not, it will be automatically interpolated.

Create and play

Also initialize the animation in setup first, but remember to refresh

// create animation
anim = skel->createAnimation("myanim", 6);
anim->setInterpolationMode(Animation::IM_SPLINE);
tracksnew = anim->createNodeTrack(lknee->getHandle(), lknee);
ent->refreshAvailableAnimationState();

//animation
as = ent->getAnimationState("myanim");
as->setEnabled(true);
as->setLoop(false);

Next, write animation frames directly in the main rendering thread, and write while rendering

// frame rendering
int i = 0;
bool MyTestApp::frameRenderingQueued(const FrameEvent &evt){   
	i++;
	TransformKeyFrame *newKF = tracksnew->createNodeKeyFrame(i);
	Quaternion quat;
	quat.FromAngleAxis(Degree(i%2? 0.0f: -90.0f), Vector3::UNIT_X);
	newKF->setRotation(lknee->getOrientation().Inverse() * quat);
	ent->refreshAvailableAnimationState();
	std::cout << as->getTimePosition() << std::endl;

	as->addTime(0.033333);
    return true;
}

There is a problem to pay attention to here. The rendering starts from the 0th frame, but if you directly modify the 0th frame, this value will not take effect until the end of the rendering process, that is to say, the frame modified in the rendering thread must be rendered in the current frame. It will take effect only after completion, so the frame you modify must be behind the current rendering frame, so the above code, the first frame modified directly, is not the first frame modified like the animation is created first and then played.

Modify joint rotation directly

Very simple, has nothing to do with creating animation sequences, just set the setManuallyControlled of the relevant joints to true in setup

SkeletonInstance *skel = ent->getSkeleton();
lshoulder = skel->getBone("Humerus.L"); lshoulder->setManuallyControlled(true);
lknee = skel->getBone("Calf.L"); lknee->setManuallyControlled(true);

Then modify the bone rotation in the render thread

int i = 0;
bool MyTestApp::frameRenderingQueued(const FrameEvent &evt){   
	i++;
	Quaternion quat;
	quat.FromAngleAxis(Degree(i%2? 0.0f: -90.0f), Vector3::UNIT_X);
	
	lknee->setOrientation(quat);
    return true;
}

Because the rendering speed is too fast, you must use breakpoints to see the driving effect of each frame. The second half of the video is the result of canceling the breakpoint and driving all the time.

It is easy to find that although this method is simple, the shared bones will fail, so once you use this method to drive two sets of the same bones, you must manually synchronize, set all joints of the two sets of bones setManuallyControlled to true, remember to delete it Code for sharing bones first

for (int j = 0; j < skel->getNumBones(); j++) {
		skel->getBone(j)->setManuallyControlled(true);
		skel2->getBone(j)->setManuallyControlled(true);
	}

Then every time you modify it, you must synchronize each joint and traverse it, and synchronize the corresponding joints of the two bones.

for (int j = 0; j < skel->getNumBones(); j++) {	
		skel2->getBone(j)->setOrientation(skel->getBone(j)->getOrientation());
	}

In this way, the movement can be synchronized, and there is also no inter-frame smoothing.

Pay attention to the pit

Be sure not to set the skeleton's setManuallyControlled to true in the frame animation driving method, otherwise each frame is driven based on the result of the previous frame. The normal skeletal animation should be similar to BVH animation, and each frame should be independent. , and based on the transformation of the initial pose, such as A-pos or T-pos, assuming that the animation frame-driven method enables manual control, then the animation result is:

postscript

This blog post records the method in which multiple bones share the same set of actions in the work. At the same time, this method supports real-time driving. For example, after calculating the rotation amount through 3D key points, it is immediately rendered.

In the future, the body driving method in unity and Unreal Engine should be updated, mainly to communicate the 3D key points extracted by deep learning between the engine and python through socket communication, and then use FABRIK or other dynamic methods to drive the virtual character. If you are interested, you can pay attention.

This blog post is updated to the WeChat public account synchronously. If you are interested, you can pay attention to it. The code can be found in the github of the WeChat public account. If you have any questions, please send a private message to the public account.

Tags: Computer Vision ogre

Posted by michealholding on Wed, 11 May 2022 19:47:19 +0300