Problem

Icon Adaptation Problem

Problem statement

We are given multiple datasets each has one picture for each lable. Pictures in one dataset has similar styles.
Now given a small subset of a new dataset, generate pictures for each lable with similar styles in this dataset.

Input

心血来潮买了手环4NFC版

结果竟然用不了NFC，要我重新注册账号？？？但我小米账号显示的是中国。

找了半天，终于在互联网某角落发现解决方案：

小米手环是华米的，数据库不互通。所以需要重新注册的是华米账号，所以到这里来注销自己的华米账号再注册就好了。

https://user.huami.com/hm_account/2.0.0/index.html?loginPlatform=web&platform_app=com.xiaomi.hm.health&v=3.7.8#/

Background

• Solving low resource domain adaptation.
• Assume has few lable for Target Domain.

Structure

• the ultimate goal of our approach is to use the mapped Source → Target samples ($X_{S→T}$ ) to
  augment the limited data of the target domain ($X_T$ ).

• let $M_S$ and $M_T$ be the task specific models trained on domains $P_S(X, Y)$ and $P_T (X, Y)$.

Relaxed Cycle Consistency

.

In supervised case

Similarly, there's

In unsupervised case

Performance

$MNIST$ domain is limited to only 10 samples per class, denoted $MNIST-(10)$

硬核机器学习课

首先介绍什么是Compressed Sensing（压缩感知）

这一段参考自如何理解压缩感知(compressive sensing)？

• 在现有的传统的信号处理模式中，信号要采样、压缩然后再传输，接收端要解压再恢复原始信号。采样过程要遵循奈奎斯特采样定理，也就是采样速率不能小于信号最高频率的两倍，这样才能保证根据采样所得的信息可以完整地恢复出原始信号。压缩感知在接收端通过合适的重构算法就可以恢复出原始信号，因此可以避免在传统的信号处理模式中的数据浪费和资源浪费问题。
• 压缩感知最初提出时，是针对稀疏信号x，给出观测模型$y=\Phi x$时，要有怎么样的$\Phi$，通过什么样的方式可以从$y$中恢复出$x$。（PS：稀疏信号，是指在这个信号x中非零元素的个数远小于其中零元素的个数。）
• Tao他们还推导了Restricted Isotropic Property (RIP) (可以理解成空间映射能基本保持稀疏向量长度，“伸缩度”很小)等一系列理论，证明了为了达到完美恢复，采样矩阵和信号稀疏度需要满足的条件和相互之间的关系。
• 然而，很多信号本身并非稀疏的，比如图像信号。此时可以通过正交变换Ψ’，将信号投影到另外一个空间，而在这个空间中，信号$a=\Psi*x$(analysis model)变得稀疏了。然后我们可以由模型$y=\Phi*a$，即$y=Φ*Ψ*x$，来恢复原始信号x。(PS:正交变换——对一个由空间$R^n$

Background

Tile is clear, "Few-Shot Adversarial Domain Adaptation"

• They need labels for new domain, but only a few.

Structure

• Definition of Pairs:

• Trainning steps

DCD means domain-class discriminator(tells it's G1,G2,G3 or G4)

• loss1: classfication loss
• loss3: adversarial loss for discriminator
• loss4: adversarial loss for generator( mixup (G1,G2) and (G3,G4))

Perfomance

* FADA - n stands for our method when we use n labeled target samples per category in training*

Inspiration

The model trained on large data with more noise have more ability to adapte other domains.

Introduction

A muti-domain image-to-image translation model.

Model Structure

Loss consist of Adversarial Loss(D tries to distinguish whether a image x from domain C is real or not, while G tries to cheat it), Domain Classification Loss (D tries to tell where a image $x_{b}$ is from, while $G_{x->c}$ tries to let D tell the fake image is from $c$ instead of $b$), Reconstruction Loss for Generator.

The structure is adapted from CycleGAN.

Trainning

When trainning one domain, set other domains to be 0.

Performance

Background

Doing the same Domain Adaptation job as CYCADA, translate images between GTA5 and Cityscapes.

This is published after CYCADA.

Method

The main difference is that there's a Embedding space in the middle.

The loss consis of 6 parts:

• Normal Classfication (P1)

• Reconstruct part (P2)

• Discriminator z part (P3)

• Discriminator x,y part (P4)

• Cycle loss (P5)

• Complex Classfication (P6)

In total, there's 3 discriminator $D_z,D_x,D_y$ and 2 encoder $f_x,f_y$ and 2 decoder $g_x,g_y$ and a classifier $h$.

Performance

State of art performance when this paper was published.

In job above the encoder is LeNet.

Someting interesting: "Switching

Aims

To match the joint distribution $P(Y_s,X_s)$ with $P(Y_t,X_t)$.

Solution

Find a matrix $A$ to let $P(A^T X_s), P(A^T X_t)$ as close as possible.
So is $P(Y_s|A^T X_s), P(Y_t|A^T X_t)$.

For the first pair, we use a method called TCA to minimize

this equals to

where $M_0$ is

For the next part, because we don't have $Y_t$, so we train a classfier C to learn

Background

We have $(X_s,Y_s)$ and $X_t$, but $X_s, X_t$ are not from the same ditribution, i.e. $P(Y_s|X_s) \neq P(Y_t|X_t)$. To solve this, we use the following method.

Assumption

There exist a transformation $\phi$ that $P(Y_s|\phi(X_s)) \approx P(Y_t|\phi(X_t))$.

Solutioin

To find $\phi$, we try to minimize following distance called MMD(maximum mean discrepancy).

To solve

要打Kaggle可惜手上没有很强的服务器怎么办呢？

Google Colab！ 良心首选

方案一

• 上传数据到Google Drive，然后Colab Link到Google Drive即可。

• 从Kaggle下载数据(校园网流量-1.3G TAT)

• 解压Kaggle数据(......此处等待10分钟)
• 上传数据到Google Drive(....................................预计还剩11个小时)
• 弃疗

方案二

• Colab上使用Kaggle API，直接在Google 服务器上下载+解压数据。

• 从Kaggle -> Account -> Create New API Token

• 往Colab 填入以下内容

!mkdir ~/.kaggle
import json
token = {"username":"username","key":"yourtoken"}
with open('/content/.kaggle/kaggle.json', 'w') as file:
    json.dump(token, file)
!cp /content/.kaggle/kaggle.json ~/.kaggle/kaggle.json
!chmod 600 /root/.kaggle/kaggle.json
!kaggle config set -n path -v{/content}

• 然后你就可以开始下载并解压数据集啦

!kaggle competitions download -c severstal-steel-defect-detection -p /content
!ls
!unzip -d ./test test_images.zip >/dev/null 2>&1
!unzip -d ./train train_images.zip >/dev/null 2>&1
!ls

• 下载（准备去玩局游戏看看什么时候能下完，等等什么？！ 133MB/S ？！）这就下完了？？？！！！
• 解压 10s 没到（还是服务器SSD NB啊，笔记本里的TLC真实垃圾）