UE4_iClone_FBXPipeline for MMD & Unity Characters

Welcome to Hirospot Personal Site on FC2

Click!

3DXchange FBX Pipeline to Unreal Engine 4
(Work in Process)

Demo Video

(Not Ready)

Previewing Unity & MMD Characters Dance in UE4

3DXchange6 Pipeline test series 4 (25) : 2015. 8. 22 Hirospot (Personal Site)

3D Character Animation made by any 3D Tools can be imported into UE4 by using FBX pipeline , however, the Character Data Format created in the 3D Tools are not always compatible.

UE4Angela2

The VMD motion for MMD characters which are converted to bvh in LiveAnimation by righthand cordinate and Mirror option, can be converted to iMotion in 3DXchange.

3DXchange can verify the FBX compatibility in a visible form of T-pose Characterizing, thus send the mesh and Animation through 'Unreal Preset' correctly.

UE4 creates 'Skeltal Mesh' and 'Skelton' for the Imported Mesh FBX, and 'Animation Sequences' for the Imported Animation FBX.

Different from iClone, UE4 includes all kind of assets for a Project within one Browser.

Animation Motion of Unity and MMD characters with extra bones are tested and previewed in the UE4 environment in this report.

Japanese Notations in the MMD character bones are found not acceptable at this moment.

Access	Provided by	Referenced Sites	Produced by
URL	E/日/中/韓	FBX Content Pipeline	unrealengine.com
URL	E/日/中/韓	FBX コンテンツ　パイプライン	unrealengine.com
URL	English	How to Create, Start, Open Your First New Project & Launch Unreal Editor [Tutorial]	WorldofLevelDesign
URL	English	22 Animation Videos	unrealengine.com
URL	English Forum	Thread: Iclone5 animations with UE4	unrealengine.com
URL	日本語	Let's Enjoy Unreal Engine UE4でゼロからユニティちゃんを表示させるまで	id:alwei
URL	English Video	Unreal Engine 4: Import Models from Unity	MandiSmash
URL	English Video	MotionBuilder to Unreal Engine, Character Animation Retargeting Tutorial	Truebones
URL	Music Video	MMD FNAF-Bo Peep Bo Peep [FULL][+Download]	Sestina Awashina
URL	English Video	iClone Pipeline Tutorial - Exporting iClone Props and Non-Human Characters to UDK	Reallusion
URL	English Video	UE4 Matinee & Blueprints Tutorial	TechteamGB
URL	English Video	UE4 Tutorial: Matinee-Adding Sound	Bridid Costello
URL	E/日/中/韓	Matinee-How To... Play Animations and Sounds	Unreal Engine 4 Documentation
URL	English Video	UE4 Tutorial 2 - Ambient sounds, animations and blueprints	Eric Lehmann
URL	日本語	Unreal Engine 4 の歩き方　第１０章サウンド	出村　成和
URL	E/日/中/韓	Unreal Engine 4 Documentation/ Editing Animation Sequences	unrealengine.com
URL	English Video	Basic Unreal Light Tutorial	335Razor
URL	日本語&ビデオ	UE4キャラを可愛く見せるマテリアル作成のコツ
URL	English/日本語	[UE4]Matineeでカットシーン制作（２）カメラを動かす、切り替える	historia

Tools	Version and Manufacturers
Unreal Engine 4	Ver 4.8.3, EPIC GAMES
iClone 3DXchange 6 Pipeline	Ver 6.11.0609, Reallusion
Unity 4	Ver 4.6.0, Unity Technologies ApS
PMDEditor	ver 0.1.3.8, by極北P
LiveAnimaiton	Version Beta 4.64, DreamFactory (url)

Contents	Manufacturers and Creators Thank you for the using
Unity-Chan	UNITY-CHAN! OFFICIAL WEBSITE UnityChan.unitypackage （Download） Candy Rock Star (C86-Unitychan) （Download）
【MMD】Model	Angela Balzac（url）、modeling:kakomiki, Windows100% magazine DVD-ROM(No.19 MMD講座)
Music	Bo Peep Bo Peep [Full] by T-ara 　YouTube
Test Motion	MMD FNAF Bo Beep Bo Beep by keeprefrigeratedfulhttp (url)
MMD Stage	水中椿ステージ.pmx　by YOUCHA (url)

Pipeline test for Unity Character by iClone Motion(.iMotion)

Sample Character	Sample Test Motion
Unity-chan.fbx (characterized in 3DXchange)	Air dance.iMotion (iClone)

First Test by Unity-chan.fbx (characterized in the former report).

1-1

Open Unity-chan.fbx in 3DXchange.

Unity-chan.fbx was made through the Report #18J. （This model is characterized.）
UnityChan

1-2

At 3DXChange Export, select Target Tool: GemeEngine_Unreal.

1-3

Set the Destination Folder and push 'OK' button to export the model with no animation.

Note: When selecting Unreal for Mesh only, Character must not have animation.
FBXBaseMesh

1-4

Confirm the exported file as 'v5_UnityChan.fbx' （4.83KB）.

1-5

Next, import some sample animations into Motion Library from iClone Template.

iClone Template	Sample Motion
	T_Pose.iMotion
G6 Motion_Heidi	Air dance.iMotion
	Catwalk.iMotion
	Stand talk.iMotion

（Sample animations shown in the Motion Library）
ImportAnimation

1-6

Export the Model again with Animation Included.

1-7

Confirm the Destination Folder.

Animation data are added under the first base mesh data.

UE4 Internationalization (Japanese to English)

2-1

Double Click 'Epic Games Launcher' button.

2-2

Click 【起動　Unreal Engine 4.8.3】orange button to start UNLEAL EDITOR.

2-3

When 'Project Browser' is shown, select 'NewProject' tab, then, select 'Blank Project', then click 'Create Project' button.

プロジェクトブラウザが現れたら、「新規のプロジェクト」タブを選択後、Blank「空のプロジェクト」を選んで
【プロジェクトを作成】Create Projectボタンをクリック。

2-4

When 'Minimal_Default' tab is shown, Select English at Editor Preferences/General/Region & Language/ Internationalization/Language.
Save and restart for English Notation.

'Minimal_Default'　タブが表示されたら、編集/エディタの環境設定/地域&言語のInternationalization/言語でEnglishを選択し、保存する。
再起動後英語表示となる。

エディタの環境設定　（Japanese）

Editor Preferences (English)
RegeonLanguage

2-5

’Unreal Project Browser’ is shown.

2-6

Select 'New Project' tab, select 'Blank' and click 'Create Project.'

Note: Project File Location should be: D/User/xxxxx/Document/Unreal Projects

2-7

MyProject/Minimal_Default is shown. Click Content Browser icon or (Cont+Shift+F).

Importing the Sample Unity Model into UE4

3-1

Push 'Import' button for the base mesh 'v5_UnityChan.fbx.

FBX Import Options	When the base mesh is imported, a new skeleton for UE4 use will be created.
	Warning: Missing Bones

3-2

Following Content Browser is shown.

3-3

Click icon for 'v5 UnityChan'. Mesh is prevewed in the Viewport as shown below.

(Need to modify Cheeks)
Preview

3-4

Preview image can be manipulated by the several method.

Preview Start	Preview Start/Stop	Stop	Stop
Mouth LeftClick		Push 'Reverse' button Twice	Click PC screen

	Mouse	Perspective
mouse Left click & move	forward/ backword	zoom in/ zoom out
mouse Left click & move	rightward/ leftward	ccw/ cw
mouse Right click & move	forward/ backward	downward/ upward
mouse Right click & move	rightward/ leftward	ccw/ cw

	Mouse		Perspective
push Alt	mouse Left click & move	forward/ backword	backward/ toward
	mouse Left click & move	rightward/ leftward	ccw/ cw
	mouse Right click & move	forward/ backword	zoom out/ zoom in
	mouse Right click & move	rightward/ leftward	zoom in/ zoom out

Key	Perspective
	Zoom In
	Zoom Out
	To the Left
	To the Right

Mouse Wheel	Perspective
rotate forward	zoom in
rotate backward	zoom out
push rotate	rotate

Importing the Animation fbx files

4-1

Click 'Import' at the Content Browser, 'FBX Import Options' panel is shows up.

4-2

Check the ’Import Animations’ and click Import button.

(The Following Warning is shown.)
Warning2

4-3

The Imported Sample Animation is shown on the Content Browser. (Orange Color)

4-4

Click the 'v5_UnityChan Airdance'. Animation is previewed in the Viewport as shown below.

(Animation is previewed properly, but need to modify Cheeks)
PreviewAnimation

Material Modification for Cheeks

5-1

Open Material Editor Browser, DoubleClick on the Mesh 0005

DoubleClick on the Mesh0005	Go to Window/Details	RightClick on the Content Browser, push 'Material'	Material Editor does not open for cheek (Body_0002)
			,
			Push Element 2 (RootNode7) at Mesh Details

5-2

Go to Detail/Material/Blend Mode, select Translucent to make Opacity adjustment.

5-3

Material Graph is used to edit the Material Expression Nodes and the main Node.

5-4

Make Connection between Texture Sample and RootNode_7 for Base Color and Opacity.

Select Translucent, connect the lines as follows, then push 'Apply' button.
TextureSample

Cheeks are corrected as shown in the Viewport.
GoodCheeks

(Note) Eyeballs material need to edit later. →　C-2

Pipeline Test for MMD Characters and VMD Motions

Sample MMD Character	Sample VMD Motion
【MMD】Angela Balzac	Bo Peeo Bo Peep(Full).vmd

Creating the vmd animation motion with T-Pose

1-1

Open LiveAnimation and click icon to load the sample character.

1-2

At File/Load Motion, select (STN_T_POSE.lam) in the Motion folder.

1-3

At File/Import/VMD, import 'Bo Peeo Bo Peep(Full).vmd'. Go to File/Save motion and save it.

1-4

At Window, Open Motion List and ActionMixer, locate the T-Pose file in front of the Motion file.

Minimal frames are needed for T-pose, delete about 90% of frames by using the 'RemoveClip' at ActionMixer Edit.

1-5

Move Pointer to 5th frame at T-pose, and push "Cut Clip", then select the rest part, use 'Delete' at Edit pull down in Action Mixer.

1-6

Push "RealTime" button, then push "Play" to confirm the total frames.

1-7

At File/Export/BVH, check on 'RightHandCoordinate' and 'Mirror', and push OK button at BVHExportOption.

（日本語選択の場合）
BVHOptionJ

Converting bvh Motion to iMotion at 3DXchange

2-1

Go to File/Open and open 'Bo Peep_RMirror.bvh. Shown below.

2-2

Rotate the Z axis by 180 degree, check on 'lock XYZ' and scale by 1700.

2-3

Select T-Pose and finish Characterizing.

2-4

Preview and Compare the motion.

LiveAnimation: Frame# =1141
LAPreview

3DXcjange: Flame# =1141, bvh motion is properly converted to iMotion.
3DXPreview

2-5

At File/Export/Exporting iClone Content, check on 'Export Animation' and 'Export iMotion'.

2-6

Confirm the iMotion in the iClone 6.

VMD	BVH	iMotion
Bo Peep Bo Peep(FUll).vmd 1,746KB	RMirror.bvh 10,042KB	Bo Peep_RMirror.iMotion 45,679KB

Adding the New iMotion to the family Unity Character in 3DX (Case1-1)

Sample Character	Sample Motion
Candy Rock Star （C86-Unitychan）	Bo Peeo Bo Peep(Full).iMotion

C-1

Load Candy Rock Star （C86-Unitychan） this time, complete the characterizingin 3DXchange.

T-Pose Characterizing for base bone: All notations in English

3DXchange base bone	Candy Rock Star （C86_Unity-chan bone）
Head	Character1-Head
Neck	Character1_Neck
Left/Right Shoulder	Character1_Left/Right Shoulder
Spine2	Character1_Spine2
Spine1	Character1_Spine1
Spine	Character1_Spine
Hips	Character1_Hips
Left/RightUpLeg	Character1_Left/RightUpLeg
Left/RightLeg	Character1_Left/RightLeg
Left/RighFoot	Character1_Left/RighFoot

C-2

Trying the Eyeballs material modification:

Candy Rock Star Face shown in UE4: Eyeballs seems to be different.

So Edited the Diffuse data: Put white Color around (behind) Pupils. Set 'Self-Illumination'=100
EyeMaterialsMod

C-3

Mesh Export: At File/'Export to Other 3D Format', go to 'Target Tool Preset' to select Unreal.

C-4

Import the 'Bo Peep_RMirror.iMotion into theMotion Library and preview.

C-4

Go to File/Export to Other 3D Format/Export FBX.

C-5

Check Mesh/Include Geometry and Animation/Include Animation, then select 'GameEngine_Unreal' at Target Tool Preset

Importing Candy Rock Star （C86 Unity-Chan.fbx） Mesh and Animation into UE4 (Case1-2)

D-1

Imported Data is shown in the Content Browser.

Mesh: First Try
C86UnityChan

D-2

C86 Unity-chan Mesh Skelton is shown in the Viewport.

First Try

D-3

Correct Cheeks: DoubleClick on the RootNode10, show Details and Material Editor.

D-4

At Details/Material/Blend Mode, select Translucent and edit the connection at Material Editor. Click 'Apply' button.

D-5

Opacity of Cheeks is changed as shown below.

First Try

D-6

Improving the Eyes, Pale Looking face and body

6-1

Edited the Eyeball texture data by Texture editor to remove greenish color in the Sclera area.

Revised for Candy Rock Star: Eyeball material (Mesh_0001&0002) edited
CandyRockStar

6-2

Face Material Test Example: Added Texture sample for Emissive Color （same for body）

6-3

Hair Test Edit: Duplicated Hair Texture and added to Emissive Color

6-4

Body Test Edit-1: Select Subsurface for Shading Model, Add (Mix) Skin Color to Emissive Color for Body & Face.

6-5

Body Test Edit-2: Duplicate the Body texture and added Texture color to Emissive Color. （Better）

6-6

Body Test Edit-3:　(MyProject 17）　

6-7

Body Test Edit-4

6-8

Cloth Test Edit

6-9

Revised for Candy Rock Star: Eyeball, hair, face, body & Cloth edited (Face: test-2, Body:test-4)

D-7

Import C86 Unity-Chan_Anim_Bo-Peep_RMirror.FBX.

First Try

7-1

Click the 'C86 Unity-Chan_Anim_Bo_Peep RMirror' (Animation Sequence) icon to preview.

First try

7-2

Preview Image by Edited Eyes, Face, Hair, Body and Cloth Materials

Adding the iMotion to the MMD Character in the 3DX (Case2-1)

E-1

Load 【MMD】Angela Balzac, complete the characterizing. (Japanese Notation)

T-Pose Characterizing for base bone: Almost notations in Japanese

3DXchange base bone	【MMD】Angela Balzac
Head	頭
Neck	首
Left/Right Shoulder	左/右肩、左/右腕、左/右ひじ、左/右手首
Spine2	(Not Used)
Spine1	上半身２
Spine	上半身
Hips	armature
Left/RightUpLeg	左/右足
Left/RightLeg	左/右ひざ
Left/RighFoot	左/右足首

E-2

At File/'Export to Other 3D Format', go to 'Target Tool Preset' to select Unreal.

E-3

Import 'Bo Peep_RMirror.iMotion in the Motion Library.

E-4

Preview the motion and go to File/Export to Other 3D Format/Export FBX.

E-5

Select Mesh and Animation, then select GameEngine_Unreal at Target Tool Preset

Importing 【MMD】 Angela.fbx Mesh and Animation into UE4 (Case2-2)

F-1

Imported Data for Mesh and Animation are shown in the Content Browser.

F-2

Red_Angela_Mesh: Material is not shown properly.

F-3

Red_Angela_Anim_Bo-Peep_RMirror: Body part does not move prolerly.

(Follwing Warning is shown: Japanese notation bones are all listed)

Adding the iMotion to the other MMD Character in the 3DX (Case3-1)

G-1

Change the Option at DAE file export in PMDEditor.

PMD Editor: File/Export/Collada (Plugin)	For English Notation
	Set Unchecked for English Notation

G-2

Use FBXConverter, and import into 3DXchange. Confirm the Scene Tree.

G-3

Scale by 800, set T-pose and finish Characterizing.

T-Pose Characterizing for base bone: Notations in English

3DXchange base bone	【MMD】Angela Balzac
Head	Bone-053
Neck	Bone-052
Left/Right Shoulder	Bone-031, Bone-032, Bone-034, Bone-036 Bone-010, Bone-011, Bone-013, Bone-015
Spine2	Bone-008
Spine1	Bone-007
Spine	Bone-001
Hips	Bone-000
Left/RightUpLeg	Bone-065, Bone-062,
Left/RightLeg	Bone-066, Bone-063,
Left/RighFoot	Bone-067, Bpne-064

G-4

Go to File/Export to Other 3D Format/Export FBX.

G-5

Import the 'Bo Peep_RMirror.iMotion into theMotion Library and preview.

G-6

Go to File/Export to Other 3D Format/Export FBX.

G-7

Select Mesh and Animation, then select GameEngine_Unreal at Target Tool Preset.

G-8

Character Image can be changed by editing the Diffuse texture.

Importing MMD AngelaE.fbx Mesh and Animation into UE4 (Case3-2)

Imported MMD AngelaERed.fbx is shown as follows: Body Texture offset is changed.

H-1

Imported MMD AngelaE_Red_Mesh.fbx contents are shown in the Content Browser.

H-2

Previewing the AngelaE_Red_Mesh.

Saved as Project19
AngelaE_Red_Preview

H-3

Next, AngelaE_Red_Animation.FBX, imported and shown in the Content Browser.

H-4

Click the 'AngelaE_Red_Anim_Bo_Peep RMirror' (Animation Sequence) icon to preview.

By selecting Bone, Translate, Rotate and Scale the bone is possible while previewing.
MMDAngelaEBone

H-5

Animation is shown properly, but Following Warning Message is shown.

Adding Camera, Mouse and Key Operation

I-1

At Minimal Derault, click Window/Toolbar. Click Matinee/Add Matinee.

I-2

Click Camera and type any Camera Name. (Cam1)

Cam1/Movement/FOVAngle is generated.
Cam1a

I-3

RightClick on the Space (Shown above) and click 'Add New Director Group'.

I-4

Click on the 'Movement' and confirm the Camera with new screen in the Perspective view.

(CameraActor screen is shown when Camera is selected)

I-5

Maniplating the Character in the Matinee Perspective View

W	E	R
Select and translate objects(W)	Select and rotate objects(E)	Select and scale objects(R)

	Mouse	Perspective	Key	Perspective	Wheel
Mouse Left Click & Move	forward/ backword	Zoom In/ Zoom Out		Zoom In	Forward
Mouse Left Click & Move	Rightward/ Leftward	CCW/ CW		Zoom Out	Backward
Mouse Right Click & Move	forward/ backword	Upward/ Downward		To the Left
Mouse Right Click & Move	Rightward/ Leftward	CCW/ CW		To the Right

I-6

At Content Browser, drag and Drop the AngelaE_Animation into the Perspective View.

(Chairs and Table are deleted)
AddCharacter

I-7

Previewing of the Character Animation is possible.

Rightclick on the Perspective view and select 'Play from Here'.

Start: RightClick on the Perspective Push: Play From Here

Stop: Push 'Esc' button.

Adding Imported Sound

J-1

Confirming the 'SOUND QUE' (Type 1)

1-1

Prepare the 'Bo Peep Bo Peep' Sound by .wav format.

1-2

Push 'Import' button, the Bo Peep Bo Peep.wav is shown on the Browser.

1-3

Right Click on the Sound icon and Push 'Create Cue'.

1-4

DoubleClick the Sound Que icon, push 'Play Que' button to get the SoundCue output.

Confirming the MATINEE (Type 2)

2-1

Select 'Matinee' at Toolbar, click 'Add Matinee' button to open MatineeActor.

2-2

RightClick on the Track Section, push 'Add New Empty Group' and type 'BoPeepTest'.

Type 'BoPeepTest' on the New Group

2-3

RightClick on the 'BoPeepTest' Folder and add 'Sound' Track.

2-4

Click the Sound Que icon on the Browser, set the start point and push Add key button.

Click

2-5

Push 'Play' button for Sound test. The Sound stops in 5 sec.

2-6

Extend the bar by moving ending marker to play the sound to the end (=230sec).

How to sync the Imported Animation and Imported Sound.

K-1

Confirm the iClone timeline for synchronizing the imported animation and sound.

Start Motion at the Frame 122 with reference to the Audio.
iCloneTimeline

Motion starts 2 sec later from Sound Start.

		Start		Stop
Sound Bo Peep Bo Peep.wav	Frame	0			13724
Sound Bo Peep Bo Peep.wav	Time (sec)	0			230
Motion Bo Peep RMirror.iMotion	Frame		122	13503
Motion Bo Peep RMirror.iMotion	Time (sec)		2	226

K-2

Using 'PlaySound' notify to sync the Sound with the Animation.

K-3

RightClick on the Track List to show the Notify and select 'PlaySound'.

K-4

Select the 'Bo Peep Bo Peep.wav' in Anim Notify/Sound first.

K-5

RightClick on the Bo-Peep letter, then, LeftClick to type the Notify time.

K-6

Push Play button and confirm the Sound and Animation in sync.

Adding the MMD Stage via 3DXchange FBX Export

L-1

Import MMD Stage （水中椿ステージ.pmx）. Scale by 800-900. Adjust Material parameters.

Select the Stage with Japanese Notation to avoid material mapping error when Character Mesh is added.

Material Name English	Material Name Japanese	Opacity	Self-Illumination	Specular
Material-04	Material-柱	50	100	0
Material-00	Material-歯車・棒	20	100	0
Material-01	Material-壁	100	50	0
Material-02	Material-天井	100	0	0
Material-03	Material-床	100	0	0

Note: Confirm that the Material names should not be the same
between Stage and Character to avoid the confusion in UE4.

L-2

At File/Go to Other 3D Format/Export FBX, select GameEngine_Unreal at Target Tool Preset.

L-3

At UE4/Minimal_Default, open Toolbar under Window Tab. Import the MMDStage.

L-4

Push 'Content' then 'Import' button to import the MMDStage.

L-5

Imported MMDStage Mesh is shown on the UE4 ContentBrowser.

(Material: Japanese name)
UE4ContentBrowser_J

(Material: English name)
UE4ContentBrowser

L-6

DoubleClick on the Stage_Mesh icon to open preview image.

L-7

Select Translucent at Material/Blend Mode, Make Opacity Connection for Material-02 (柱)

Use Multiply for the Base Coror and Opacity. Push 'Apply' button.
Maaterial-02a

Adding Character(s) and Anim Track

(Note) Creating the animation track in a group is
only possible by assigning the skeltal mesh actor.

M-1

Click Characters with Animation in the Content Browser and drag into Perspective Viewport.

M-2

Click the Character in the Viewport.

M-3

Click 'Matinee' icon on the Toolbar and open 'MatineeActor'. Open 'Window' to find Tracks.

MatineeActor

M-4

RightClick on the Track Section, click 'Add New Skeltal Group' and type the Character Name.

M-5

RightClick on the Character Name, and select 'Add New Sound Track' in the Actors Pane.

M-6

Move the Time Carson at 2 sec position, select Anim track.

M-7

Click on the Anim Track, push Add Key, then click the Bo\ Peep Animation sequence.

→

M-8

Animation data name is shown in the Timeline bar.

M-9

Move the Time slider to 0 sec position, select 'Sound Cue' at Content Browser and push Add Key.

（RightClick on the Character Name, then select Actors/Add Selected Actors on the Popup.）

Camera Operation

N-1

Make a 'MainCamera' Group, push 'Add Key' and select MainCamera in the pulldown.

N-2

Add Director Group to use the Camera view.

N-3

Add Cameras to change the scene. Use 'Add Keys' at the point to change camera.

N-4

Use Top/Left/Right views to locate the camera position and Character.

N-5

Push 'Stop' button in the MateneeActor when to switch Perspective/Left view.

N-6

Adjust the FOV(Field of View) value at the Details/CameraActor/Camera Settings/Field of View.

Camera FOV is shown in pink line. Use RightClick to shift the screen.
TopView

FOV (Field of View) value settings
Details

N-7

Select the Track and push 'Add Key' when the Camera and Chracter factors are updated.

N-8

Click and drag the Unity-chan into the Perspective Viewport.

Persona Animation Editor

Work in Process