Chainer_ a Flexible and Intuitive Framework for Complex Neural Networks Presentation

6 789%'$$: !; 0- ?%14%+(%) @A4B> @CDE

! #$%&'($% #)*+%,-). /-) 0-+1$%& 2%3)*$ 2%4,-).5 !"#"$% '%(#

)*%#+ ,#-.$

/*$.$**$( 0$12#*345 6+78

Simple neural networks: “Just deep” convolutional neural network

AlexNet, Kryzyevsky+, 2012 ImageNet winner

GoogLeNet, Szegedy+, 2014 ImageNet winner

Residual Network, He+, 2015 ImageNet winner

Complex neural networks: Stochastic, densely-connected, hierar hierarchical, chical, recurr recurrent ent

Stochastic Residual Net, Huang+, 2016

Dense CNN, Huang+, 2016

FractalNet, Larsson+, 2016

Recurrent NN (LSTM)

http://chainer.org/ !

9":%+$* %4 #+$ #. 1"$ 0;6<6=>4?@@#*1$(
!

9":%+$* ?4$4 : ?+%C?$ 47"$B$ +:B$( !"#$"%&'%()$ 9":%+$* D14 E$*F 2$-- 1# 7#B@-$G +$?*:- +$12#*34

Preferred Networks (PFN) A startup that applies deep learning to industrial IoT !

Founded: March 2014

!

Headquarter: Tokyo, Japan

!

US subsidiary: SF bay area, California

!

Company size: 50 engineers & researchers

!

Investors: Toyota, FANUC, NTT Manufacturing

Healthcare

 Automotive

Deep learning

Industrial IoT

What we do with Chainer: Partnering with world-leading companies !

!

=@@-F%+H ($$@ -$:*+%+H 1# %+(?41*%:- @*#I-$B4 2%1" *$:->2#*-( (:1: ̶

!@$7%D7 *$C?%*$B$+145 B#(%D7:J#+ 1# :-H#*%1"B45 B:+F 1*%:->:+(>$**#*4

̶

9#B@-$1$-F (%K$*$+1 .*#B B:3%+H H$+$*:->@?*@#4$ *$7#H+%J#+ 4F41$B4

= @*##.L /M0 2#+ 1"$ N+( @*%O$ P =B:O#+ /%73%+H 9":--$+H$ NQRS

Toyota

NTT

FANUC

Panasonic Cisco NVIDIA

Two types of background behind DL frameworks R8 !7:-:I%-%1F>#*%$+1$( !

!

!

T4$>7:4$4 %+ B%+(

N8 M-$G%I%-%1F>#*%$+1$( !

T4$>7:4$4 %+ B%+(

̶

6B:H$U4@$$7" *$7#H+%J#+ 4F41$B

̶

0$2 :-H#*%1"B *$4$:*7"

̶

M:41
̶

YZ< @*#[$714 .#* =6 @*#(?714

/*#I-$B 1F@$

!

/*#I-$B 1F@$

̶

= .$2 H$+$*:- :@@-%7:J#+4

̶

;:*%#?4 4@$7%D7 :@@-%7:J#+4

̶

RQV B%--%#+ 1*:%+%+H 4:B@-$4

̶

RQV 3 1*:%+%+H 4:B@-$4

̶

RQV +#($4 7-?41$* 2U .:41 +$12#*3

̶

R +#($ 2%1" B?-J@-$ \/T4

/#44%I-$ I#W-$+$73

!

/#44%I-$ I#W-$+$73

̶

X?+%+H #. 2$-->3+#2+ :-H#*%1"B4

̶

X*%:->:+(>$**#* %+ @*#1#1F@%+H

̶

<%41*%I?1$( 7#B@?1:J#+ .#* B#($-U(:1:>@:*:--$- 1*:%+%+H

̶

<$I?HH%+H5 @*#D-%+H Z *$.:71#*%+H

̶

]2:%1 JB$ (?*%+H 7#B@%-:J#+^

Designed for efficient research & development

!

M-$G%I-$L +$2 3%+(4 #. 7#B@-$G B#($-4 .#* E:*%#?4 :@@-%7:J#+4

!

6+1?%JE$L *:@%( @*#1#1F@%+H :+( $_7%$+1 1*%:->:+(>$**#*

!

/#2$*.?-L 7#B@:*:I-$ @$*.#*B:+7$ .#* R +#($ Z B?-J>\/T4

TensorFlow Scalability-oriented

Ca! e  

Theano

Torch Flexibility-oriented

Agenda !

<$$@ -$:*+%+H .*:B$2#*3 I:4%74

!

`:4%74 #. 9":%+$*

!

9":%+$* .$:1?*$4

!

/$*.#*B:+7$ :+( :@@-%7:J#+4

Neural network and computation

Forward computation

Image

Object: Tulip

x1 ・ ・

h1 ・ ・

・ ・ ・ ・

Sensor ・ ・

k1

y1 ・

・ ・

・

・ ・

kM

Anomaly score: 0.35

yM

hH

xN

Category: Sports

Text

Input

Hidden units

Output

Backward computation (backpropagation)

Building and training neural networks: Computational graph construction is the key R8

N8

a8

9#+41*?71 : 7#B@?1:J#+:- H*:@" ̶

`:4$( #+ +$12#*3 ($D+%J#+ H%E$+ IF ?4$*4

̶

9":%+4 #. .?+7J#+4 :+( #@$*:J#+4 #+ %+@?1 E:*%:I-$4

9#B@?1$ -#44 :+( H*:(%$+14 ̶

M#*2:*( 7#B@?1:J#+ 1# 7:-7?-:1$ -#44 .#* : B%+%I:17"

̶

`:73@*#@:H:J#+ H%E$4 H*:(%$+14 1# :-- #. @:*:B$1$*4

)@JB%O$ B#($̶

T@(:1$ $:7" @:*:B$1$* 2%1" 1"$ H*:(%$+1

̶

Y$@$:1 ?+J- 7#+E$*H$+7$

Step 1. is the most important and there are many approaches

Building blocks !

X"$4$ .?+7J#+:-%J$4 :*$ E$*F 4%B%-:* I$12$$+ .*:B$2#*34

!

`?1 1"$ 41*?71?*$5 :I41*:7J#+ -$E$-5 :+( %+1$*.:7$ :*$ (%K$*$+1

!

61 7#B$4 1# 1"$ ($4%H+ #. (#B:%+>4@$7%D7 -:+H?:H$ .#* 00

 Array data structure (vector/matrix/tensor)

Network (computational graph)

Operations & functions

Optimizer (SGD/AdaGrad/Adam)

3 types of domain-specific language: Network definition to computational graph construction !

X$G1
!

̶

bG8 9:K$ ]@*#1#1G1^

̶

bG8 90Xc ]0
ii <$D+%J#+ %+ 1$G1 .L j k=lL k;:*%:I-$l5 k`lL k;:*%:I-$l5 k9lL mk`l5 khl5 k=ln5 k*$1lL mk9l5 kVl5 Rn o d 9#B@%-$ . e 7#B@%-$]k.81G1l^ ( e .]=e+@8#+$4]RQ^5 `e+@8#+$4]RQ^ h N^

!FBI#-%7 @*#H*:B

!

6B@$*:JE$ @*#H*:B

̶

)@$*:J#+4 #+ 4FBI#-4

̶

<%*$71 7#B@?1:J#+4 #+ *:2 (:1: :**:F4

̶

bG8 X"$:+#

̶

bG8 X#*7"8++

̶

bG8 X$+4#*M-#2 ̶ bG8 9":%+$* Ex. MXNet

d !FBI#-%7 ($D+%J#+ = e ;:*%:I-$]f=g^ ` e ;:*%:I-$]f`g^ 9e`h= < e 9 V 9#+41:+1]R^ d 9#B@%-$ . e 7#B@%-$]<^ ( e .]=e+@8#+$4]RQ^5 `e+@8#+$4]RQ^ h N^

d 6B@$*:JE$ ($7-:*:J#+ : e +@8#+$4]RQ^ I e +@8#+$4]RQ^ h N 7eIh: (e7VR

Comparison of DSL type F?G 4:1%

X$G1
   A    !    <        :    +    *    $    1    +    6

!FBI#-%7

6B@$*:JE$

H)-5I

0-<5I

• '?B:+>*$:(:I-$ ($D+%J#+ • 0#+>@*#H*:BB$* 7:+ $:4%-F $(%1 1"$ +$12#*3

• T4$*4 B?41 41?(F 1"$ .#*B:1 • M#*B:1 B%H"1 ":E$ 1# I$ $G1$+($( .#* +$2 :-H#*%1"B4

• !1:J7 :+:-F4%4 :1 7#B@%-$

• T4$*4 B?41 41?(F 4@$7%:-

• )@JB%O:J#+ I$.#*$ 1*:%+%+H • b:4F 1# @:*:--$-%O$

4F+1:G

• p:F +$$( B#*$ $K#*14 1# %B@-$B$+1 +$2 :-H#*%1"B4

• A$44 $K#*14 1# -$:*+ 4F+1:G

• ':*( 1# #@JB%O$ %+ :(E:+7$

• b:4F ($I?HH%+H :+( @*#D-%+H • A$44 $_7%$+1 %+ B$B#*F • !?%1:I-$ .#* +$2 :-H#*%1"B4

:--#7:J#+ :+( @:*:--$-%O:J#+

2%1" 7#B@-$G -#H%7

9":%+$* %4 :1 1"$ $G1*$B$ $+( #. %B@$*:JE$ @*#H*:B .#* "%H" q$G%I%-%1F

Agenda !

<$$@ -$:*+%+H .*:B$2#*3 I:4%74

!

`:4%74 #. 9":%+$*

!

9":%+$* .$:1?*$4

!

/$*.#*B:+7$ :+( :@@-%7:J#+4

Chainer as an open-source project !

"W@4LUUH%1"?I87#BU@.+$1U7":%+$*

!

rs 7#+1*%I?1#*4

!

R5SSt 41:*4 Z uRs .#*34

!

v5sQv 7#BB%14

!

=7JE$-F ($E$-#@$( ̶

aN *$-$:4$4 .*#B ER8Q8Q ]w?+$ NQRv^ 1# ER8RS8Q ]!$@1$BI$* NQRS^

̶

=E$*:H$ d7#BB%14 @$* 2$$3 x rQ

Chainer software stack !

9":%+$* %4 I?%-1 #+ 1#@ #. 0?B/F :+( 9T<=

!

9?/F %4 :-4# %+1*#(?7$( :4 :+ $C?%E:-$+1 #. 0?B/F #+ \/T

9":%+$* 9?/F 0?B/F

7?<00

`A=!

9T<=

9/T

0;6<6= \/T

Graph build scheme (1/2) - Define-and-Run: Most of frameworks use this (Chainer does not) !

<$D+$L I?%-( : 7#B@?1:J#+:- H*:@" I:4$( #+ ($D+%J#+

!

Y?+L ?@(:1$ 1"$ B#($- ]@:*:B$1$*4^ ?4%+H 1*:%+%+H (:1:4$1

<$D+$ 0$12#*3 ($D+%J#+

/:*:B$1$*4 9#B@?1:J#+:H*:@"

\*:(%$+1 .?+7J#+

=?1# (%K$*$+J:J#+

Y?+ X*:%+%+H (:1:

/:*:B$1$*4 9#B@?1:J#+:H*:@"

T@(:1$ \*:(%$+1 .?+7J#+

A#44 Z H*:(%$+1

Graph build scheme (2/2) - Define-by-Run: Computational graph construction on the fly !

0# H*:@" %4 7#+41*?71$( I$.#*$ 1*:%+%+H

!

6+41$:(5 1"$ H*:@" %4 I?%-1 :1 $:7" .#*2:*( 7#B@?1:J#+

!

9#B@?1:J#+:- H*:@" 7:+ I$ B#(%D$( (F+:B%7:--F .#* $:7" %1$*:J#+U4:B@-$ #* ($@$+(%+H #+ 4#B$ 7#+(%J#+4

<$D+$>IF>Y?+ p#($($D+%J#+ X*:%+%+H (:1:

/:*:B$1$*4 9#B@?1:J#+:H*:@"
T@(:1$ \*:(%$+1 .?+7J#+

Define-by-Run example: MLP for MNIST !

)+-F 1*:+4.#*B:J#+4 I$12$$+ ?+%14 :*$ 4$1 I$.#*$ 1*:%+%+H l1 = Linear(784, n_units) l2 = Linear(n_units, 10))

!

9#++$7J#+ %4 H%E$+ :4 .#*2:*( 7#B@?1:J#+ def forward(x): h1 = ReLU(l1(x)) return l2(h1)

W

bias

W

bias ０

ｘ Linear l1

ReLU

y

h1 Linear l2

５ ９

Define-by-Run: An interpreted language for neural network !

=(E:+1:H$ ̶

M-$G%I%-%1F .#* +$2 :-H#*%1"B4 2%1" 7#B@-$G 7#B@#+$+14 "

̶

6+1?%JE$ 7#(%+H 2%1" "%H"-F %B@$*:JE$ +$12#*3 ($D+%J#+ "

!

bG8 *$7?**$+15 *$7?*4%E$5 ($+4$-F>7#++$71$(5 (%-:1$(5 $17

bG8 41#7":4J7 +$12#*3 #. 2"%7" H*:@" 7":+H$4 .#* $:7" %1$*:J#+

9?**$+1 (*:2I:734 ̶

\*:@" %4 H$+$*:1$( $E$*F JB$ :-4# .#* DG$( +$12#*34

̶

0# #@JB%O:J#+ $E$+ .#* 41:J7 @:*1 #. H*:@"4 "

w6X>-%3$ :+:-F4%4 :+( 4?IH*:@" 7:7"$ B%H"1 I$ ?4$.?-

Backpropagation through computational graph !

9#+4%($* :+ #I[$7JE$ ]A%+38A%+$:*^L

!

X"%4 7#B@?1$4 1"$ E:-?$ #. A %+ .#*2:*( 7#B@?1:J#+5 :+( 4%B?-1:+$#?4-F I?%-(4 1"$ .#--#2%+H 7#B@?1:J#+:- H*:@"

ｘ

W

b

*

+

L = f(x * w + b)

%4 ;:*%:I-$ %4 M?+7J#+

f

!

X"$ H*:(%$+1 #. A 7:+ I$ 7#B@?1$( 2%1" *$4@$71 1# :+F E:*%:I-$4 IF I:73@*#@:H:J#+

!

X"$+ 1"$ #@JB%O$* ?@(:1$4 1"$ E:-?$ #. @:*:B$1$*4

L

Code sample (1/3): Convolutional neural network class AlexNet(Chain): def __init__(self): super(AlexNet, self).__init__( conv1=L.Convolution2D(3, 96, 11, stride=4), conv2=L.Convolution2D(96, 256, 5, pad=2), conv3=L.Convolution2D(256, 384, 3, pad=1), conv4=L.Convolution2D(384, 384, 3, pad=1), conv5=L.Convolution2D(384, 256, fc6=L.Linear(9216, 4096), fc7=L.Linear(4096, 4096),

conv2d conv2d

3, pad=1),

fc8=L.Linear(4096, 1000), ) def __call__(self, x, t): h = F.max_pooling_2d(F.relu( F.local_response_normalization(self.conv1(x))), 3, stride=2) h = F.max_pooling_2d(F.relu( F.local_response_normalization(self.conv2(h))), 3, stride=2) h = F.relu(self.conv3(h)) h = F.relu(self.conv4(h)) h = F.max_pooling_2d(F.relu(self.conv5(h)), 3, stride=2) h = F.dropout(F.relu(self.fc6(h)), train=self.train) h = F.dropout(F.relu(self.fc7(h)), train=self.train) y = self.fc8(h) return y

conv2d conv2d conv2d linear linear linear

* ImageNet Classification with Deep Convolutional Neural Networks

Code sample (2/3): Recurrent neural network class SimpleRNN(Chain): def __init__(self, n_vocab, n_units): super(SimpleRNN, self).__init__( embed=L.EmbedID(n_vocab, n_units) x2h=L.Linear(n_units, n_units), h2h=L.Linear(n_units, n_units), h2y=L.Linear(n_units, n_vocab),) self.h = None def __call__(self, x): y, h_new = self.fwd_one_step(x, self.h) self.h = h_new return y def fwd_one_step(self, x, h): x = F.tanh(self.embed(x)) if h is None: h = F.tanh(self.x2h(x)) else: h = F.tanh(self.x2h(x) + self.h2h(h)) y = F.softmax(self.h2y(h)) return y, h

# Truncated BPTT (length=3) for i in range(0, datasize, batchsize): ... accum_loss += model(x, t) if i % bptt_length == 0: model.zerograds() accum_loss.backward() accum_loss.unchain_backward() optimizer.update() 3/451 6789

;.<588./1 =1>1.

:51451



!"#

$

%"#



!"&

$

%"&



!"'

$

%"'



!"(

$

%"( )*++ -./01$ 2 '

Code sample (3/3): Stochastic Residual Network !

= E:*%:+1 #. Y$4%(?:- 0$1 1":1 43%@4 7#++$7J#+4 41#7":4J7:--F ̶

)?1@$*.#*B$( 1"$ #*%H%+:- Y$4%(?:- 0$1 ]6B:H$0$1 NQRv 2%++$*5 p!Y^

̶

!1#7":4J7 43%@L w/ survival probability: # Mock code in Chainer class StochasticResNet(Chain): def __init__(self, prob, size, …): super(StochasticResNet, size, …).__init__( ## Define f[i] as same for Residual Net )

self.p = prob # Survival probabilities def __call__(self, h): for i in range(self.size): b = numpy.random.binomial(1, self.p[i]) Taken from http://arxiv.org/abs/1603.09382v2 G. Huang et al.

c = self.f[i](h) + h if b == 1 else h h = F.relu(c)

Agenda !

<$$@ -$:*+%+H .*:B$2#*3 I:4%74

!

`:4%74 #. 9":%+$*

!

9":%+$* .$:1?*$4

!

/$*.#*B:+7$ :+( :@@-%7:J#+4

Easy to debug during forward computation # MNIST input has

784 dimensions,

not

748

def __init__(self): super(MLP3Wrong, self).__init__( l1=L.Linear(748, 100), l2=L.Linear(100, 100), l3=L.Linear(100, 10) )

Where the error actually happened in forward computation

Type is checked at the numpy array level

Error stack trace in IPython (Will be more clean soon)

CuPy: (partially-)NumPy-compatible GPU library !

!

p#JE:J#+L 0?B/F V 9T<= e 9?/F ̶

0?B/F %4 1"$ 41:+(:*( -%I*:*F %+ /F1"#+ .#* +?B$*%7:- 7#B@?1:J#+

̶

T+.#*1?+:1$-F 0?B/F (#$4 0)X 2#*3 2%1" 9T<=5 %+41$:( 9?/F (#$4 %1

bG8 9/TU\/T>:H+#4J7 -#H4?B$G@ .?+7J#+ def logsumexp(x, axis=None): xp = cuda.get_array_module(x) #Get CuPy or NumPy x_max = x.max(axis) exp_sum = xp.exp(x - x_max).sum(axis) return x_max + xp.log(exp_sum)

!

9?/F 4?@@#*14L ̶

0;6<6= 7?`A=! :+( 7?<00 .#* .:41$* 7#B@?1:J#+

̶

T4$*>($D+$( .?+7J#+4 :+( 9T<= 3$*+$-4

̶

=**:F %+($G%+H5 4-%7%+H5 1*:+4@#4$5 :+( *$4":@$

̶

=-- (1F@$45 I*#:(7:4J+H5 B$B#*F @##-5 $17

Use CuPy just like NumPy !

9#+E$*4%#+ I$12$$+ numpy.ndarray :+( cupy.ndarray w_c = cupy.asarray(numpy.ones(10)) # cupy.ndarray w_n = cupy.asnumpy(cupy.ones(10)) # numpy.ndarray

!

!@$$(>?@ IF 9?/F #E$* 0?B/FL v 1# Nv JB$4 def test(xp): a = xp.arange(1000000).reshape(1000, -1) return a.T * 2

+5%= for i in range(1000): test(numpy) for i in range(1000): test(cupy)

0?B/F

51%%J 31

N5tNt

R8Q

9?/F

vsv

v8Q

9?/F V p$B#*F /##-

RNa

Na8s

6+1$- 9#*$ %r>urtQ Pa8SQ\'O5 aN\`5 \$M#*7$ \Xy trQ

Training loop abstraction (from v1.11.0): Manual training loop is not necessary !

<:1:4$1 Z 61$*:1#*L (:1:4$1 B:+%@?-:J#+ #E$* %1$*:J#+4

!

X*:%+$* Z T@(:1$*L $G$7?1$ 1*:%+%+H -##@4 2U 4$z+H4

!

bG1$+4%#+L (# 4#B$1"%+H $-4$ (?*%+H 1*:%+%+H -##@4 ̶

bG8 (?B@{H*:@"]^5 4+:@4"#15 A#HY$@#*15 /*#H*$44`:*5

Training dataset & loop settings

Reporting for each epoch

Miscellaneous !

)1"$* .$:1?*$4 ̶

6+41:-- 2%1" @%@ %+ #+$ -%+$L $ pip install chainer

̶

p?-J>\/T 4?@@#*1 IF $G@-%7%1-F 4$-$7J+H 1"$ 6< 1# ?4$

̶

/*$>1*:%+$( 9:K$ B#($- %B@#*1 .*#B p#($- |##

̶

p#($- 4$*%:-%O:J#+ Z 4:E$ Z -#:( L '
̶

=?1#B:J7 %+%J:-%O:J#+ #. -:F$* %+@?1 4%O$ 2"$+ 4@$7%D$( :4 +,$"

Agenda !

<$$@ -$:*+%+H .*:B$2#*3 I:4%74

!

`:4%74 #. 9":%+$*

!

9":%+$* .$:1?*$4

!

/$*.#*B:+7$ :+( :@@-%7:J#+4

Benchmark results (CNN): Chainer shows comparable performance !

M#*2:*( 7#B@?1:J#+ %4 :-B#41 1"$ 4:B$ 2%1" X#*7"UXM

!

9":%+$*g4 I:732:*( 7#B@?1:J#+ ":4 I$$+ :77$-$*:1$( .*#B ER8r 1# ER8Rv 1# :7"%$E$ 7#B@:*:I-$ @$*.#*B:+7$ Forward computation (msec)

'#!! '!!!

'#!!

9/6:; 9*<=/67)/>

&!!

Backward computation (msec)

?;8@<*6 5'A'BC

'!!!

9*<=/67)/> &!!

?;8@<*6 5'AD %!!

?8E* F<8G5*H

%!!

?;8@<*6 5'A'BC ?;8@<*6 5'AD ?8E* F<8G5*H

$!!

$!!

#!!

#!!

!

!

()*+,*-

9/6:;

.// 1*,*2..3( 45*67*8()*+,*.//01*,*2..3( 45*67*8Chainer v1.15* shows our uno #cial result measured independently on September 2016

Chainer in industry: From demonstrations to commercialization !

9":%+$*>I:4$( 7#--:I#*:J#+4 :*$ #+>H#%+H 2%1" @:*1+$*4 ̶

!

7#B@?1$* E%4%#+5 ($$@ *$%+.#*7$B$+1 -$:*+%+H5 $17}

M=0T9 2%-- 7#BB$*7%:-%O$ 1"$ #?17#B$ IF 1"$ $+( #. NQRS Bin-picking robot w/ deep learning based recognition

Toyota: Self-learning autonomous cars

FANUC FIELD system Panasonic: Parking space detection

Recent achievement: PFN used Chainer at Amazon Picking Challenge 2016 !

9":%+$* .#* %B:H$ 4$HB$+1:J#+ :+( #I[$71 *$7#H+%J#+

!

/M0 1$:B 2#+ 1"$ N+( @*%O$ .#* @%73%+H :+( u1" .#* 41#2 1:434 ̶

9#B@$1$ 2%1" #1"$* Rv $41:I-%4"$( *#I#J74 -:I4 :+( 7#B@:+%$4

̶

)+-F a B#+1"4 1# I?%-( 1"$ E%4%#+ 4F41$B :+( *#I#1 ":+( .*#B 47*:17"

Stow task

Pick task

1. 214: Delft 2. 186: NimbRo 3. 164: MIT 4. 161: PFN … 16. ARCV

1. 105: Delft 2. 105: PFN 3. 97: NimbRo 4. 67: MIT … 16. Rutgers

Researcher’s perspective on Chainer: Easy to use and extend !

Fast Learning curve ̶

!

Expertise

Concise object model, modular libraries

Straightforward GPU support ̶

Numpy and CuPy interchangeable

̶

Support for multiple GPUs

!

Easily customizable

!

Easy to debug / fix

!

Latest deep learning algorithms

Time

Researcher’s perspective on Chainer: Example case study !

Solving for systems of equations for a physical, thermodynamics system

!

Several methods can be possible:

!     %     " #&      #'              #()* $ "      $              

̶

Physical modeling

̶

Empirical modeling / other machine learning techniques

̶

Deep learning

!

Prior knowledge about monotonic and coupling properties can be more easily incorporated into deep learning using Chainer

!

In this situation, the problem was easily set-up using Chainer and the model was both expressive and concise