光学 精密工程, 2019, 27 (8): 1836, 网络出版: 2020-01-19
基于改进CycleGAN的图像风格迁移
Image style transfer based on improved CycleGAN
摘要
图像风格迁移是用风格图像对指定图像的内容进行重映射, 利用GAN自动进行图像风格迁移, 可减少工作量, 且结果丰富。特定情况下GAN方法所用的配对数据集很难获得。为了避免利用传统GAN进行图像风格迁移受到成对数据集的限制, 提高风格迁移效率, 本文利用改进的循环一致性对抗网络CycleGAN实现图像风格迁移, 用密集连接卷积网络DenseNet代替原来网络生成器的深度残差网络ResNet, 用同一映射损失和感知损失组成的损失函数度量风格迁移损失。所做改进使网络性能得到了提升, 取消了网络对成对样本的限制, 提高了风格迁移生成图像的质量。同时进一步提高了稳定性, 加快了网络收敛速度。论文所提方法对建筑图像进行了风格迁移, 实验结果表明, 生成图像的PSNR值平均提高了6.27%, SSIM值均提高了约10%。因此, 本文提出的改进的CycleGAN图像风格迁移方法生成的风格图像效果更优。
Abstract
Image style transfer exploits a specified style to modify given image content. An automatic image style transfer based on a Generative Adversarial Network (GAN) can reduce the workload and yield rich results. In some cases, the pair datasets required by the classical GAN were difficult to obtain. To overcome the limitations of paired datasets by a traditional GAN and improve the efficiency of style transfer, this study proposed an image style transfer method based on an improved Cycle-consistent adversarial network (CycleGAN). In this study, the deep residual network adopted by the conventional network generator was replaced by the dense connection convolution network, and a novel loss function composed of the same mapping and perceptual losses was used to measure the style transfer loss. These improvements were shown to increase the network performance, overcome the networks limitations on paired samples, and improve the quality of images generated by style migration. In addition, the stability was further improved and the network convergence speed was accelerated. Experiments demonstrate that the peak signal-to-noise ratio of the image generated by the proposed method increase 6.27% on average, where as the structural similarity index measure increased by approximately 10%. The improved CycleGAN image style transfer method proposed in this study can thus generate better style images.
杜振龙, 沈海洋, 宋国美, 李晓丽. 基于改进CycleGAN的图像风格迁移[J]. 光学 精密工程, 2019, 27(8): 1836. DU Zhen-long, SHEN Hai-yang, SONG Guo-mei, LI Xiao-li. Image style transfer based on improved CycleGAN[J]. Optics and Precision Engineering, 2019, 27(8): 1836.