# Visualizing the Latent Space of Vector Drawings from the Google QuickDraw Dataset with SketchRNN, PCA and t-SNE

This is the third part in a series of notes on my exploration of the recently released Google QuickDraw dataset 1, using the concurrently released SketchRNN model.

The QuickDraw dataset is curated from the millions of drawings contributed by over 15 million people around the world who participated in the "Quick, Draw!" A.I. Experiment, in which they were given the challenge of drawing objects belonging to a particular class (such as "cat") in under 20 seconds.

SketchRNN is an impressive generative model that was trained to produce vector drawings using this dataset. It was of particular interest to me because it cleverly assembles many of the latest tools and techniques recently developed in machine learning, such as Variational Autoencoders, HyperLSTMs (a HyperNetwork for LSTM), Autoregressive models, Layer Normalization, Recurrent Dropout, the Adam optimizer, among others.

# Exploring the Google QuickDraw Dataset with SketchRNN (Part 3)

This is the third part in a series of notes on my exploration of the recently released Google QuickDraw dataset 1, using the concurrently released SketchRNN model.

The QuickDraw dataset is curated from the millions of drawings contributed by over 15 million people around the world who participated in the "Quick, Draw!" A.I. Experiment, in which they were given the challenge of drawing objects belonging to a particular class (such as "cat") in under 20 seconds.

SketchRNN is an impressive generative model that was trained to produce vector drawings using this dataset. It was of particular interest to me because it cleverly assembles many of the latest tools and techniques recently developed in machine learning, such as Variational Autoencoders, HyperLSTMs (a HyperNetwork for LSTM), Autoregressive models, Layer Normalization, Recurrent Dropout, the Adam optimizer, among others.

# Exploring the Google QuickDraw Dataset with SketchRNN (Part 2)

This is the second part in a series of notes on my exploration of the recently released Google QuickDraw dataset, using the concurrently released SketchRNN model.

In the previous note, we set up our development environment, downloaded a subset of the data along with some pre-trained models, and developed some utilities for visualizing the data in the notebook. We retain most of the code from previous note and omit the expository code and markdown cells.

The QuickDraw dataset is curated from the millions of drawings contributed by over 15 million people around the world who participated in the "Quick, Draw!" A.I. Experiment, in which they were given the challenge of drawing objects belonging to a particular class (such as "cat") in under 20 seconds.

SketchRNN is a very impressive generative model that was trained to produce vector drawings using this dataset. It was of particular interest to me because it cleverly combines many of the latest tools and techniques recently developed in machine learning, such as Variational Autoencoders, HyperLSTMs (a HyperNetwork for LSTM), Autoregressive models, Layer Normalization, Recurrent Dropout, the Adam optimizer, and others.

In [48]:
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

The autoreload extension is already loaded. To reload it, use:

In [49]:
import matplotlib.pyplot as plt
import matplotlib.patches as patches

import numpy as np
import tensorflow as tf

from matplotlib.animation import FuncAnimation
from matplotlib.path import Path
from matplotlib import rc

from six.moves import map

In [50]:
from magenta.models.sketch_rnn.sketch_rnn_train import \
reset_graph,
PRETRAINED_MODELS_URL)
from magenta.models.sketch_rnn.model import Model, sample
from magenta.models.sketch_rnn.utils import (lerp,
slerp,
get_bounds,
to_big_strokes,
to_normal_strokes)

In [52]:
# For inine display of animation
# equivalent to rcParams['animation.html'] = 'html5'
rc('animation', html='html5')

In [53]:
# set numpy output to something sensible
np.set_printoptions(precision=8,
edgeitems=6,
linewidth=200,
suppress=True)

In [54]:
tf.logging.info("TensorFlow Version: {}".format(tf.__version__))

INFO:tensorflow:TensorFlow Version: 1.1.0


#### Getting the Pre-Trained Models and Data¶

In [7]:
DATA_DIR = ('http://github.com/hardmaru/sketch-rnn-datasets/'
'raw/master/aaron_sheep/')
MODELS_ROOT_DIR = '/tmp/sketch_rnn/models'

In [8]:
DATA_DIR

Out[8]:
'http://github.com/hardmaru/sketch-rnn-datasets/raw/master/aaron_sheep/'
In [9]:
PRETRAINED_MODELS_URL

Out[9]:
'http://download.magenta.tensorflow.org/models/sketch_rnn.zip'
In [10]:
download_pretrained_models(
models_root_dir=MODELS_ROOT_DIR,
pretrained_models_url=PRETRAINED_MODELS_URL)

INFO:tensorflow:/tmp/sketch_rnn/models/sketch_rnn.zip already exists, using cached copy
INFO:tensorflow:Unzipping /tmp/sketch_rnn/models/sketch_rnn.zip...
INFO:tensorflow:Unzipping complete.


We look at the layer normalized model trained on the aaron_sheep dataset for now.

In [11]:
MODEL_DIR = MODELS_ROOT_DIR + '/aaron_sheep/layer_norm'

In [12]:
(train_set,
valid_set,
test_set,
hps_model,
eval_hps_model,

INFO:tensorflow:Downloading http://github.com/hardmaru/sketch-rnn-datasets/raw/master/aaron_sheep/aaron_sheep.npz
INFO:tensorflow:Dataset combined: 8000 (7400/300/300), avg len 125
INFO:tensorflow:model_params.max_seq_len 250.
total images <= max_seq_len is 7400
total images <= max_seq_len is 300
total images <= max_seq_len is 300
INFO:tensorflow:normalizing_scale_factor 18.5198.

In [222]:
class SketchPath(Path):

def __init__(self, data, factor=.2, *args, **kwargs):

vertices = np.cumsum(data[::, :-1], axis=0) / factor
codes = np.roll(self.to_code(data[::,-1].astype(int)),
shift=1)
codes[0] = Path.MOVETO

super(SketchPath, self).__init__(vertices,
codes,
*args,
**kwargs)

@staticmethod
def to_code(cmd):
# if cmd == 0, the code is LINETO
# if cmd == 1, the code is MOVETO (which is LINETO - 1)
return Path.LINETO - cmd

In [69]:
def draw(sketch_data, factor=.2, pad=(10, 10), ax=None):

if ax is None:
ax = plt.gca()

x_min, x_max, y_min, y_max = get_bounds(data=sketch_data,
factor=factor)

sketch = SketchPath(sketch_data)

patch = patches.PathPatch(sketch, facecolor='none')


#### The real fun begins¶

Everything up to here has more or less been copied straight from the previous notebook. Now we load the pre-trained SketchRNN model and use it to begin our exploration of the test dataset.

In [110]:
# construct the sketch-rnn model here:
reset_graph()
model = Model(hps_model)
eval_model = Model(eval_hps_model, reuse=True)
sample_model = Model(sample_hps_model, reuse=True)

INFO:tensorflow:Model using gpu.
INFO:tensorflow:Input dropout mode = False.
INFO:tensorflow:Output dropout mode = False.
INFO:tensorflow:Recurrent dropout mode = True.
INFO:tensorflow:Model using gpu.
INFO:tensorflow:Input dropout mode = False.
INFO:tensorflow:Output dropout mode = False.
INFO:tensorflow:Recurrent dropout mode = False.
INFO:tensorflow:Model using gpu.
INFO:tensorflow:Input dropout mode = False.
INFO:tensorflow:Output dropout mode = False.
INFO:tensorflow:Recurrent dropout mode = False.

In [111]:
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())

In [112]:
# loads the weights from checkpoint into our model

INFO:tensorflow:Loading model /tmp/sketch_rnn/models/aaron_sheep/layer_norm/vector.
INFO:tensorflow:Restoring parameters from /tmp/sketch_rnn/models/aaron_sheep/layer_norm/vector


The helper functions for encoding a sketch to some latent code $z$ and then decoding it back to a sketch were provided in the original notebook. I just made some minor syntactic changes and removed the behaviour of plotting as a side-effect.

In [73]:
def encode(input_strokes):
strokes = to_big_strokes(input_strokes).tolist()
strokes.insert(0, [0, 0, 1, 0, 0])
seq_len = [len(input_strokes)]
z = sess.run(eval_model.batch_z,
feed_dict={
eval_model.input_data: [strokes],
eval_model.sequence_lengths: seq_len})[0]
return z

In [74]:
def decode(z_input=None, temperature=.1, factor=.2):
z = None
if z_input is not None:
z = [z_input]
sample_strokes, m = sample(
sess,
sample_model,
seq_len=eval_model.hps.max_seq_len,
temperature=temperature, z=z)


Now we get a random sample from the test dataset

In [120]:
sketch = test_set.random_sample()

In [138]:
fig, ax = plt.subplots(figsize=(3, 3),
subplot_kw=dict(xticks=[],
yticks=[],
frame_on=False))

draw(sketch, ax=ax)

plt.show()


We project it into the 128-dimensional latent space using the pre-trained encoder

In [183]:
z = encode(sketch)
z.shape

Out[183]:
(128,)

Now we can reconstruct the original sketch from the learned latent representation using the pre-trained decoder, with temperature $\tau=0.8$. The temperature parameter controls the level of randomness in the samples generated by the model, which becomes deterministic as $\tau \to 0$, and produces samples that are the most likely point in the probability density function. See pg. 7 of the original paper for further discussion of the effects the temperature parameter has on the sampling process.

In [216]:
sketch_reconstructed = decode(z, temperature=.6)
sketch_reconstructed.shape

Out[216]:
(250, 3)
In [217]:
fig, ax = plt.subplots(figsize=(3, 3),
subplot_kw=dict(xticks=[],
yticks=[],
frame_on=False))

draw(sketch_reconstructed, ax=ax)

plt.show()


#### Variance in the Reconstruction¶

The grid of drawings below consists of samples of the reconstructed drawings at various settings of the temperature parameter. The first column is the original drawing, and each of the remaining columns are 5 samples of the reconstructed drawing with $\tau$ increasing from 0.1 to 0.9.

In [182]:
fig, ax_arr = plt.subplots(nrows=5,
ncols=10,
figsize=(8, 4),
subplot_kw=dict(xticks=[],
yticks=[],
frame_on=False))
fig.tight_layout()

for row_num, ax_row in enumerate(ax_arr):
for col_num, ax in enumerate(ax_row):
if not col_num:
draw(sketch, ax=ax)
xlabel = 'original'
else:
t = col_num / 10.
draw(decode(z, temperature=t), ax=ax)
xlabel = r'$\tau={}$'.format(t)
if row_num+1 == len(ax_arr):
ax.set_xlabel(xlabel)

plt.show()


At the lowest setting of the temperature at $\tau=0.1$, we see the samples consistently share a similar appearance - they all look like vertical strokes emanating from a fluffy cloud. However, they are also consistently dissimilar to the original sketch. In this sense, the samples from the models seems to exhibit high bias and low variance. As we increase the variance in the samples by increasing $\tau$, we start to find some samples that resemble our original sketch. But when we increase $\tau$ a little too much, beyond say 0.8, we begin to see a little too much randomness in the samples.

#### Drawing Comparisons¶

Humans typically write and, by extension, draw from left to right, top to bottom. Here, I wanted to animate the process of the original sketch being drawn alongside the decoder's reconstruction of the sketch to compare stroke patterns, typical stroke lengths, etc.

In [218]:
fig, (ax1, ax2) = plt.subplots(ncols=2, nrows=1, figsize=(6, 3),
subplot_kw=dict(xticks=[],
yticks=[]))
fig.tight_layout()

(x_min_1,
x_max_1,
y_min_1,
y_max_1) = get_bounds(data=sketch, factor=.2)

(x_min_2,
x_max_2,
y_min_2,
y_max_2) = get_bounds(data=sketch_reconstructed, factor=.2)

x_min = np.minimum(x_min_1, x_min_2)
y_min = np.minimum(y_min_1, y_min_2)

x_max = np.maximum(x_max_1, x_max_2)
y_max = np.maximum(y_max_1, y_max_2)

ax1.set_xlabel('Original')

ax2.set_xlabel('Reconstruction')

Out[218]:
<matplotlib.text.Text at 0x7f232a2097d0>
In [219]:
def animate(i):

original = SketchPath(sketch[:i+1])
reconstructed = SketchPath(sketch_reconstructed[:i+1])

facecolor='none'))

facecolor='none'))

return patch1, patch2

In [220]:
frames = np.maximum(sketch.shape[0],
sketch_reconstructed.shape[0])
frames

Out[220]:
249
In [221]:
FuncAnimation(fig,
animate,
frames=frames-1,
interval=15,
repeat_delay=1000*3,
blit=True)

Out[221]:

Unfortunately, the strokes that make up a sketch have been normalized with the Ramer–Douglas–Peucker algorithm, which is a simple stroke simplification process. This means the strokes aren't quite the same as the that which the human originally used to construct the sketch. Moreover, the timing of each stroke are also important to understanding patterns in how humans draw quick sketches. While timestamp data is provided in the full QuickDraw dataset, they are not preserved in the modified version of the dataset used by SketchRNN.

# Exploring the Google QuickDraw Dataset with SketchRNN (Part 1)

This is the first part in what will hopefully be a series of notes 1 on my exploration of the recently released Google QuickDraw dataset, using the concurrently released SketchRNN model.

The QuickDraw dataset is curated from the millions of drawings contributed by over 15 million people around the world who participated in the "Quick, Draw!" A.I. Experiment, in which they were given the challenge of drawing objects belonging to a particular class (such as "cat") in under 20 seconds.

SketchRNN is a very impressive generative model that was trained to produce vector drawings using this dataset. It was of particular interest to me because it cleverly combines many of the latest tools and techniques recently developed in machine learning, such as Variational Autoencoders, HyperLSTMs (a HyperNetwork for LSTM), Autoregressive models, Layer Normalization, Recurrent Dropout, the Adam optimizer, and others.

This notebook was based on the notebook included with the code release. I've made significant stylistic changes and some minor changes to ensure Python 3 compatibility as Magenta only supports Python 2 currently.

1. These notes will likely be quite hasty and unpolished , as it has been written more for myself than anyone else. While I've always tried to avoid cluttering up my blog with notebooks on in-progress work, I've decided to just make a habit of posting them everytime I complete a session of work on something for the day. The aim is to start creating series of short and succinct notebooks, and avoid further accumulating a collection of long and disorganized notebooks that I never want to touch again, because the sheer effort of going through, cleaning up the experimental code, and articulating it effectively in a well-thought-out blog article just requires too much willpower. Doing it this way will hopefully make it much easier for me to create and share useful content quickly.

#### Environment Set-up¶

Some preamble for plotting (I really ought to put these in a config file at some point...), and importing dependencies. I've made the imports here explicit so we know exactly what methods/objects are imported and can find the module from which they were imported more easily.

In [1]:
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

In [2]:
import matplotlib.pyplot as plt
import matplotlib.patches as patches

import numpy as np
import tensorflow as tf

from matplotlib.path import Path

In [3]:
from magenta.models.sketch_rnn.sketch_rnn_train import \
reset_graph,
PRETRAINED_MODELS_URL)
from magenta.models.sketch_rnn.model import Model, sample
from magenta.models.sketch_rnn.utils import (get_bounds,
to_big_strokes,
to_normal_strokes)

In [4]:
# set numpy output to something sensible
np.set_printoptions(precision=8,
edgeitems=6,
linewidth=200,
suppress=True)

In [5]:
tf.logging.info("TensorFlow Version: {}".format(tf.__version__))

INFO:tensorflow:TensorFlow Version: 1.1.0


#### Getting the Pre-Trained Models and Data¶

In [6]:
DATA_DIR = ('http://github.com/hardmaru/sketch-rnn-datasets/'
'raw/master/aaron_sheep/')
MODELS_ROOT_DIR = '/tmp/sketch_rnn/models'

In [7]:
DATA_DIR

Out[7]:
'http://github.com/hardmaru/sketch-rnn-datasets/raw/master/aaron_sheep/'
In [8]:
PRETRAINED_MODELS_URL

Out[8]:
'http://download.magenta.tensorflow.org/models/sketch_rnn.zip'
In [9]:
download_pretrained_models(
models_root_dir=MODELS_ROOT_DIR,
pretrained_models_url=PRETRAINED_MODELS_URL)

INFO:tensorflow:/tmp/sketch_rnn/models/sketch_rnn.zip already exists, using cached copy
INFO:tensorflow:Unzipping /tmp/sketch_rnn/models/sketch_rnn.zip...
INFO:tensorflow:Unzipping complete.


The directory tree looks like this. There are a few pretrained models for us to explore.

In [10]:
!tree -L 3 /tmp/sketch_rnn/

/tmp/sketch_rnn/
└── models
├── aaron_sheep
│   ├── layer_norm
│   ├── layer_norm_uncond
│   ├── lstm
│   └── lstm_uncond
├── catbus
│   └── lstm
├── elephantpig
│   └── lstm
├── flamingo
│   └── lstm_uncond
├── owl
│   └── lstm
└── sketch_rnn.zip

14 directories, 1 file


We look at the layer normalized model trained on the aaron_sheep dataset for now.

In [11]:
MODEL_DIR = MODELS_ROOT_DIR + '/aaron_sheep/layer_norm'

In [12]:
(train_set,
valid_set,
test_set,
hps_model,
eval_hps_model,

INFO:tensorflow:Downloading http://github.com/hardmaru/sketch-rnn-datasets/raw/master/aaron_sheep/aaron_sheep.npz
INFO:tensorflow:Dataset combined: 8000 (7400/300/300), avg len 125
INFO:tensorflow:model_params.max_seq_len 250.
total images <= max_seq_len is 7400
total images <= max_seq_len is 300
total images <= max_seq_len is 300
INFO:tensorflow:normalizing_scale_factor 18.5198.


#### Drawing the Dataset¶

The strokes object variable is the list of data points, which are sequences of strokes, represented as a 2D NumPy array of xy-offsets and the pen state.

In [13]:
len(train_set.strokes)

Out[13]:
7400

We can get a random sample from the dataset like so

In [122]:
a = train_set.random_sample()
a.shape

Out[122]:
(122, 3)

In the original notebook, the authors implemented their own function to iterate through a stroke sequence and write out a SVG Path string. I found this a bit inelegant and cumbersome to work with.

Here we simply subclass Path, which underpins all matplotlib.patch classes. This is almost perfect for our data format since it "supports the standard set of moveto, lineto, curveto commands to draw simple and compound outlines consisting of line segments and splines. The Path is instantiated with a (N,2) array of (x,y) vertices, and a N-length array of path codes." We just need to normalize the data slightly as we shall explain later.

In [133]:
class StrokesPath(Path):

def __init__(self, data, factor=.2, *args, **kwargs):

vertices = np.cumsum(data[::, :-1], axis=0) / factor
codes = np.roll(self.to_code(data[::,-1].astype(int)),
shift=1)

super(StrokesPath, self).__init__(vertices,
codes,
*args,
**kwargs)

@staticmethod
def to_code(cmd):
# if cmd == 0, the code is LINETO
# if cmd == 1, the code is MOVETO (which is LINETO - 1)
return Path.LINETO - cmd


Now drawing the strokes becomes as simple as

In [134]:
fig, ax = plt.subplots(figsize=(3, 3))

strokes = StrokesPath(a)

patch = patches.PathPatch(strokes, facecolor='none')

x_min, x_max, y_min, y_max = get_bounds(data=a, factor=.2)

ax.set_xlim(x_min-5, x_max+5)
ax.set_ylim(y_max+5, y_min-5)

ax.axis('off')

plt.show()


We define this as a function to maximize modularity and reusability

In [135]:
def draw(stroke, factor=.2, pad=(10, 10), ax=None):

if ax is None:
ax = plt.gca()

x_min, x_max, y_min, y_max = get_bounds(data=stroke,
factor=factor)

strokes = StrokesPath(stroke)

patch = patches.PathPatch(strokes, facecolor='none')

ax.axis('off')


Now it is easy to fully take advantage of the functionality provided by Matplotlib to create more complex plots. For example, to draw the sketches in a grid, we just call our draw function on the grid of axes created with subplots.

In [136]:
fig, ax_arr = plt.subplots(nrows=5,
ncols=10,
figsize=(8, 4),
subplot_kw=dict(xticks=[],
yticks=[],
frame_on=False))
fig.tight_layout()

for ax_row in ax_arr:
for ax in ax_row:
strokes = train_set.random_sample()
draw(strokes, ax=ax)

plt.show()

##### Explanation¶

The last column of the 2D array is essentially a binary value that gives the pen action taken prior to the next stroke in the sequence.

In [137]:
a[::,-1].astype(int)

Out[137]:
array([1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1])

A 1 means the pen is to be lifted from the current point before being moved to the next point. In terms of SVG Path commands or Matplotlib Path codes, this corresponds to the next point being preceded by the m command or having the MOVETO code, respectively. Otherwise, the pen just draws a line to the next point, which means it is preceded by the l command or has the LINETO code.

Our to_code static method above simply turns the pen action for the next state to the corresponding Matplotlib Path code.

In [138]:
{c: getattr(Path, c) for c in dir(Path) if c.isupper()}

Out[138]:
{'CLOSEPOLY': 79,
'CURVE3': 3,
'CURVE4': 4,
'LINETO': 2,
'MOVETO': 1,
'NUM_VERTICES_FOR_CODE': {0: 1, 1: 1, 2: 1, 3: 2, 4: 3, 79: 1},
'STOP': 0}
In [139]:
StrokesPath.to_code(a[::,-1].astype(int))

Out[139]:
array([1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 1])

As explained, the $i$th code is meant for the $i+1$th vertex, so we are not done just yet. We must right shift the array by 1. But what do we do with the first and last elements of the sequence?

Note that the pen action for all stroke sequences are terminated with the pen being lifted. This corresponds to the next vertex having a MOVETO code. However, there are no vertices left, so this is superfluous and may be discarded.

In [140]:
all(StrokesPath.to_code(a[-1,-1]) == Path.MOVETO \
for a in train_set.strokes)

Out[140]:
True

On the other hand, the first vertex is always required to have a MOVETO code (otherwise, where the drawing actually begins is not well-defined). Therefore, the most simple and elegant solution is just to np.roll the array to right by 1, so the first code takes on the value of the last code, which is always a MOVETO.

Lastly, while SVG Paths supports both absolute positions and relative offsets (with M and m commands respectively), Matplotlib only supports absolute positions. As the datasets gives the points as relative offsets, we convert it to absolute positions simply by calling taking the cumulative sum with np.cumsum.

# Trigonometric functions with recursion and higher-order functions in Python

In [1]:
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

In [2]:
import matplotlib.pyplot as plt
import numpy as np

from itertools import count, islice, takewhile
from functools import reduce, partial


The Taylor series expansion for the trigonometric function $\sin{x}$ around the point $a=0$ (also known as the Maclaurin series in this case) is given by:

$$\sin{x} = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \dotsb \text{ for all } x$$

The $k$th term of the expansion is given by

$$\frac{(-1)^k}{(2k+1)!} x^{2k+1}$$

It is easy to evaluate this closed-form expression directly. However, it is more elegant and indeed more efficient to compute the terms bottom-up, by iteratively calculating the next term using the value of the previous term. This is just like computing factorials or a sequence of Fibonacci numbers using the bottom-up approach in dynamic programming.

# Matplotlib Unchained

I a previous post, I outlined how to embed a Matplotlib Animation directly in the Jupyter Notebook as a HTML5 video. In this notebook, we take the same Animation and save it as a GIF using Imagemagick. First, let us reproduce the FuncAnimation object from the notebook.

In [1]:
%matplotlib inline

In [2]:
import numpy as np
import matplotlib.pyplot as plt

from matplotlib import animation, rc
from IPython.display import HTML, Image

In [3]:
# equivalent to rcParams['animation.html'] = 'html5'
rc('animation', html='html5')

In [4]:
# Create new Figure with black background
fig = plt.figure(figsize=(8, 8), facecolor='black')

# Add a subplot with no frame

# Generate random data
data = np.random.uniform(0, 1, (64, 75))
X = np.linspace(-1, 1, data.shape[-1])
G = 1.5 * np.exp(-4 * X * X)

# Set y limit (or first line is cropped because of thickness)
ax.set_ylim(-1, 70)

# No ticks
ax.set_xticks([])
ax.set_yticks([])

# 2 part titles to get different font weights
ax.text(0.5, 1.0, "MATPLOTLIB ", transform=ax.transAxes,
ha="right", va="bottom", color="w",
family="sans-serif", fontweight="light", fontsize=16)
ax.text(0.5, 1.0, "UNCHAINED", transform=ax.transAxes,
ha="left", va="bottom", color="w",
family="sans-serif", fontweight="bold", fontsize=16)

# Generate line plots
lines = [ax.plot((1-i/200.)*X, i+G*d , color="w", lw=1.5-i/100.)[0]
for i, d in enumerate(data)]

In [5]:
def animate(*args):
# Shift all data to the right
data[:, 1:] = data[:, :-1]

# Fill-in new values
data[:, 0] = np.random.uniform(0, 1, len(data))

# Update data
for i, line in enumerate(lines):
line.set_ydata(i + G * data[i])

# Return modified artists
return lines

In [6]:
# call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(fig, animate, interval=20, blit=True)


Now, we just need to save the animation instance with writer=imagemagick. But before we do that, we first make sure imagemagick has been properly installed on our system.

In [7]:
!brew install imagemagick

Updating Homebrew...


Now we can go ahead and save it as a GIF.

In [8]:
anim.save('../../files/unchained.gif', writer='imagemagick', fps=60, savefig_kwargs=dict(facecolor='black'))


Let's read it back in and display it to make sure it saved as expected.

In [9]:
Image(url='../../../unchained.gif')

Out[9]:

# Save Matplotlib Animations as GIFs

I a previous post, I outlined how to embed a Matplotlib Animation directly in the Jupyter Notebook as a HTML5 video. In this notebook, we take the same Animation and save it as a GIF using Imagemagick. First, let us reproduce the FuncAnimation object from the notebook.

In [1]:
%matplotlib inline

In [2]:
import numpy as np
import matplotlib.pyplot as plt

from matplotlib import animation, rc
from IPython.display import HTML, Image

In [3]:
# equivalent to rcParams['animation.html'] = 'html5'
rc('animation', html='html5')

In [4]:
# First set up the figure, the axis, and the plot element we want to animate
fig, ax = plt.subplots()

ax.set_xlim(( 0, 2))
ax.set_ylim((-2, 2))

line, = ax.plot([], [], lw=2)

In [5]:
# initialization function: plot the background of each frame
def init():
line.set_data([], [])
return (line,)

In [6]:
# animation function. This is called sequentially
def animate(i):
x = np.linspace(0, 2, 1000)
y = np.sin(2 * np.pi * (x - 0.01 * i))
line.set_data(x, y)
return (line,)

In [7]:
# call the animator. blit=True means only re-draw the parts that
# have changed.
anim = animation.FuncAnimation(fig, animate, init_func=init,
frames=100, interval=20, blit=True)

In [8]:
anim

Out[8]:

Now, we just need to save the animation instance with writer=imagemagick. But before we do that, we first make sure imagemagick has been properly installed on our system.

In [9]:
!brew install imagemagick

Warning: imagemagick-7.0.4-6 already installed


Now we can go ahead and save it as a GIF.

In [10]:
anim.save('../../files/animation.gif', writer='imagemagick', fps=60)


Finally, let's read it back in and display it to make sure it saved as expected.

In [11]:
Image(url='../../../animation.gif')

Out[11]:

# Walkthrough: Deploying a Flask app with Redis Queue (RQ) Workers and Dashboard using Kubernetes

In this article, we walkthrough the steps to deploy a simple Flask app together with Redis Queue (RQ), a minimalistic job queue built on Redis, using Kubernetes to provision and manage the necessary deployments and services for our stack.

# Re-implementing the Kubernetes Guestbook Example with Flask and NGINX

The official Kubernetes walkthrough guides often points to the guestbook application as a quintessential example of how a simple, but complete multi-tier web application can be deployed with Kubernetes. As described in the README, it consists of a web frontend, a redis master (for storage), and a replicated set of redis 'slaves'.

This seemed like an ideal starting point for deploying my Flask applications with a similar stack, and also makes use of redis master/slaves. The difficulty I found with readily making use of this example as a starting point is that the frontend is implemented in PHP, which is considerably different to modern paradigms (Node.js, Flask/Django, Rails, etc.) As described in the README:

A frontend pod is a simple PHP server that is configured to talk to either the slave or master services, depending on whether the client request is a read or a write. It exposes a simple AJAX interface, and serves an Angular-based UX. Again we'll create a set of replicated frontend pods instantiated by a Deployment — this time, with three replicas.

I figured re-implementing the frontend pod in with Flask would require minimal changes - the UI would remain mostly the same, and the actual interaction with the redis master/slaves is quite trivial.

# A Better Approach For Initializing New Nikola Themes (since v7.7.5)

A few months ago, I wrote a post on Creating a Nikola theme with Sass-compiled Bootstrap. Since then, Nikola 7.7.5 has added several new features which makes it less tedious to get started with your custom theme.

## Initializing the Theme

First, I initialize a theme named tiao, which automatically creates the necessary directories and files for me.

$nikola theme --new=tiao --engine=jinja --parent=bootstrap3-jinja [2016-05-18T02:29:49Z] INFO: theme: Creating theme tiao with parent bootstrap3-jinja and engine jinja in themes/tiao [2016-05-18T02:29:49Z] INFO: theme: Created directory themes/tiao [2016-05-18T02:29:49Z] INFO: theme: Created file themes/tiao/parent [2016-05-18T02:29:49Z] INFO: theme: Created file themes/tiao/engine [2016-05-18T02:29:49Z] INFO: theme: Theme themes/tiao created successfully. [2016-05-18T02:29:49Z] NOTICE: theme: Remember to set THEME="tiao" in conf.py to use this theme.$ tree themes/tiao
themes/tiao
├── engine
└── parent

0 directories, 2 files