Publications

Selected Publications

Perceptual Image Quality Assessment (IQA)

...we are able to build machines that perform significantly better and quicker than many of our organs, like arms and legs; however, when it comes to model our eyes and brain toward human perception, the odyssey proves to be much more difficult.

For Natural Images

Since the invention of the camera in early 19th century, naturally-captured images by cameras have been the dominant mass of visual signals.

H. Wu, Z. Zhang, E. Zhang, C. Chen, L. Liao, A. Wang, C. Li, W. Sun, Q. Yan, G. Zhai, W. Lin, “Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision”, ICLR 2024 spotlight. (download)
K. Xu, L. Liao, J. Xiao, C. Chen, H. Wu, Q. Yan, W. Lin, “Boosting Image Quality Assessment through Efficient Transformer Adaptation with Local Feature Enhancement”, CVPR 2024 . (download)
H. Wu, Z. Zhang, E. Zhang, C. Chen, L. Liao, A. Wang, K. Xu, C. Li, J. Hou, G. Zhai, X. Geng, W. Sun, Q. Yan, W. Lin, “Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models”, CVPR 2024 . (download)
C. Chen, J. Mo, J. Hou, H. Wu, L. Liao, W. Sun, Q. Yan, W. Lin, “TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment”, IEEE Trans. on Image Processing, accepted. (download)
Y. Li, B. Chen, B. Chen, M. Wang, S. Wang, W. Lin, "Perceptual Quality Assessment of Face Video Compression: A Benchmark and An Effective Method", IEEE Trans. on Multimedia, 2024, accepted. (download)
H. Wu, C. Chen, L. Liao, J. Hou, W. Sun, Q. Yan, J. Gu, W. Lin, “Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment”, IEEE Trans on Pattern Analysis and Machine Intelligence, 45(12):15185-15202, 2023. (download)
X. Wang, J. Xiong, W. Lin, "Visual Interaction Perceptual Network for Blind Image Quality Assessment", IEEE Trans. on Multimedia, 25: 8958-8971 2023. (download)
Z. Wang , Q. Jiang , S. Zhao , W. Feng , W. Lin , "Deep Blind Image Quality Assessment Powered by Online Hard Example Mining", IEEE Transactions on Multimedia , in press . (download)
H. Wu, E. Zhang, L. Liao, C. Chen, J. Hou, A. Wang, W. Sun, Q. Yan, W. Lin, "Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach," ACM MM 2023.
Y. Liu, J. Wu, A. Li, L Li, W. Dong, G. Shi, W. Lin, "Video Quality Assessment with Serial Dependence Modeling ”, IEEE Transactions on Multimedia, accepted (download)
X. Wang, J. Xiong, W. Lin, "Visual Interaction Perceptual Network for Blind Image Quality Assessment", IEEE Trans. on Multimedia, in press. (download)
H. Wu, L. Liao, J. Hou, C. Chen, E. Zhang, A. Wang, W. Sun, Q. Yan, W. Lin, “Explore Opinion-Unaware Video Quality Assessment with Semantic Affinity Criterion”, IEEE ICME 2023 (download)
H. Wu, C. Chen, J. Hou, L. Liao, A. Wang, W. Sun, Q. Yan, W. Lin, “FAST-VQA: Efficient End-to-End Video Quality Assessment with Fragment Sampling”, ECCV, 2022 (download)
L. Li, P. Chen, W. Lin, M. Xu, G. Shi, "From Whole Video to Frames: Weakly-Supervised Domain Adaptive Continuous-Time QoE Evaluation", IEEE Transactions on Image Processing, 31: 4937-4951, 2022. (download)
H. Zhang, Y. Zhang, L. Zhu, W. Lin, “Deep Learning-based Perceptual Video Quality Enhancement for 3D Synthesized View”, IEEE Trans. on Circuits and Systems for Video Technology, 32(8): 5080 – 5094, 2022. (download)
Q. Jiang, Z. Liu, K. Gu, F. Shao, X. Zhang, H. Liu, W. Lin, “Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and An Objective Metric”, IEEE Trans Image Process., 31: 2279-2294, 2022 (download) (source codes)
The database gives groundtruth to answer the question: why don’t you like a picture? It can be used in further research regarding a human-like foundation model .
J. Si, B. Huang, H. Yang, W. Lin, Z. Pan, “A no-reference stereoscopic image quality assessment network based on binocular interaction and fusion mechanisms”, IEEE Transactions on Image Processing, 31: 3066 – 3080, 2022. (download)
L. Liao, K. Xu, H. Wu, C. Chen, W. Sun, Q. Yan, W. Lin, "Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment", ACM MM, oral paper, 2022 (download)
P. Chen, L. Li, J. Wu, Y. Zhang, W. Lin , "Temporal Reasoning Guided QoE Evaluation for Mobile Live Video Broadcasting", IEEE Transactions on Image Processing, 30:3278-3292, 2021.(download)
J. Ma, J. Wu, L. Li, W. Dong, X. Xie, G. Shi, W. Lin, "Blind Image Quality Assessment with Active Inference”, IEEE Transactions on Image Processing, 30(3): 3650-3663, 2021. (download)
J. Wu, W. Yang, L. Li, W. Dong, G. Shi, W. Lin, “Blind Image Quality Prediction with Hierarchical Feature Aggregation”, Information Sciences, 552:167-182, 2021. (download)
M. Karimi, M. Nejati, W. Lin, “Bi-Disparity Sparse Feature Learning for 3D Visual Discomfort Prediction”, Signal Processing, 188, Nov 2021. (download)
K. Gu, X. Xu, J. Qiao, Q. Jiang, W. Lin, D. Thalmann, “Learn a Unified Blind Image Quality Metric via On-Line and Off-Line Big Training Instances”, IEEE Transactions on Big Data, 6(4): 780 - 791, 2020. (download)
...simultaneously coping with natural scene & screen content images
J. Wu, J. Ma, F. Liang, W. Dong, G. Shi, W. Lin , “End-to-End Blind Image Quality Prediction with Cascaded Deep Neural Network”, IEEE Transactions on Image Processing, 29: 7414-7426, 2020 (download)

D. Li, T. Jiang, W. Lin, M. Jiang, “Which Has Better Visual Quality: The Clear Blue Sky or a Blurry Animal?”, IEEE TRANSACTIONS ON MULTIMEDIA， 21(5): 1221 - 1234, 2019.
Most conventional objective metrics prefer blurry animals (relatively complex visual content) over clear blue sky (simple content), contradicting with human perception: content-aware features help... (Paper & Source Codes)

●
J. Wu, J. Zeng, W. Dong, G. Shi, W. Lin, “Blind Image Quality Assessment with Hierarchy: Degradation From Local Structure to Deep Semantics”, Journal of Visual Communication and Image Representation, 58: 353-362, 2019. (download )
●

J Wu, Y Liu, W Dong, G Shi, W Lin, "Quality Assessment for Video with Degradation Along Salient Trajectories", IEEE Transactions on Multimedia, 21 (11), 2738-2749, 2019. (download)
●

Q. Li, W. Lin, K. Gu, Y. Zhang, Y. Fang, “Blind Image Quality Assessment Based on Joint Log-Contrast Statistics”, Neurocomputing, 331:189-198, 2019. (download)
●
J. Wu, M. Zhang, G. Shi, X. Xie, W. Lin, “No-reference image quality assessment with visual pattern degradation“, Information Sciences, 504: 487-500, 2019. (download)
●

B. Hu, L. Li, H. Liu, W. Lin, J. Qian, "Pairwise-Comparison-based Rank Learning for Benchmarking Image Restoration Algorithms", IEEE Transactions on Multimedia, 21(8): 2042 – 2056, 2019. (download)
●

●

K. Gu, D. Tao, J-F Qiao, W. Lin, “Learning a No-Reference Image Quality Assessment Model of Enhanced Images with Big Data”, IEEE Transactions on Neural Networks and Learning Systems, 29(4): 1301-1313, 2018. (download)
An attempt for automatic IQA-based image enhancement via big (unlimited) data and unsupervised training
●

F. Shao, Z. Zhang, Q. Jiang, W. Lin, G. Jiang, “Towards domain transfer for no-reference quality prediction of asymmetrically distorted stereoscopic images”, IEEE Trans. Circuits and Systems for Video Technology, 28(3): 573-585, 2018. (download)
●

Q. Jiang, F. Shao, W. Lin, G. Jiang, “Learning a referenceless stereopair quality engine with deep nonnegativity constrained sparse autoencoder”, Pattern Recognition, 76: 242-255, 2018. (download)
●

Q. Jiang, F. Shao, W. Lin, K. Gu, G. Jiang, H. Sun, “Optimizing Multi-Stage Discriminative Dictionaries for Blind Image Quality Assessment”, IEEE TRANSACTIONS ON MULTIMEDIA, 20(8): 2035-2048, 2018. (download)
●

Y. Zhu, S. Guntuku, W. Lin, G. Ghinea, J. Redi, “Measuring Individual Video QoE: A Survey, and Proposal for Future Directions Using Social Media”, ACM Transactions on Multimedia Computing Communications and Applications, 14(2s), Article No. 30, May 2018. (download)
●

T. J. Liu, K. H. Liu, J. Y. Lin, W. Lin, C.-C. Jay Kuo, "A ParaBoost Method to Image Quality Assessment", IEEE Transactions on Neural Networks and Learning Systems, 28(1): 107-121, 2017. (download)
An extension to the multi-methodfusion (MMF) earlier (IEEE TIP 2013 paper below): basic image quality scorers & auxiliary image quality scorers are built as weak scorers, for scorer selection.
●

S. G. Wang, C. Deng, W. Lin, G. Huang, B. Zhao, “NMF-based Image Quality Assessment Using Extreme Learning Machine”, IEEE Transactions on Cybernetics, 47(1): 232 - 243, 2017. (download)
●

H. Wang, J. Fu, W. Lin, S. Hu, C.-C. Jay Kuo, L. Zuo, “Image Quality Assessment Based on Local Linear Information and Distortion-Specific Compensation”, IEEE Transactions on Image Processing, 26(2): 915–926, 2017. (download)
●

F. Shao, W. Lin, Z. Li, G. Jiang, Q. Dai, “Toward Simultaneous Visual Comfort and Depth Sensation Optimization for Stereoscopic 3-D Experience”, IEEE Transactions on Cybernetics, 47(12): 4521-4533, 2017. (download)
●

L. Li, W. Xia, W. Lin, Y. Fang, S. Wang, “No-Reference and Robust Image Sharpness Evaluation Based on Multi-scale Spatial and Spectral Features”, IEEE Transactions on Multimedia, 19(5): 1030-1040, 2017. (download)
●

Q. Li, W. Lin, Y. Fang, “BSD: Blind Image Quality Assessment Based on Structural Degradation”, Neurocomputing, 236:93-103, 2017. (download)
●

L. Xu, J. Li, W. Lin, Y. Zhang, L. Ma, Y. Fang, Y. Yan, “Multi-task Rank Learning for Image Quality Assessment”, IEEE Trans. Circuits and Systems for Video Technology, 27(9): 1833 - 1843, 2017. (download)
●

F. Shao, W. Tian, W. Lin, G. Jiang, Q. Dai, "Learning sparse representation for no-reference quality assessment of multiply-distorted stereoscopic images", IEEE Transactions on Multimedia,19(8): 1821 - 1836, 2017. (download)
●

K. Gu, W. Lin, G. Zhai, X. Yang, W. Zhang, C. W. Chen, “No-Reference Quality Metric of Contrast-Distorted Images Based on Information Maximization”, IEEE Transactions on Cybernetics, 47(12): 4559 - 4565, 2017. (download)
●

K. Gu, L. Li, H. Lu, X. Min, W. Lin, “A Fast Reliable Image Quality Predictor by Fusing Micro- and Macro-Structures”, IEEE Transactions on Industrial Electronics, 64(5): 3903 - 3912, 2017. (download)
●

Q. Li, W. Lin, J. Xu, Y. Fang, “Blind Image Quality Assessment Using Statistical Structural and Luminance Features”, IEEE Trans. Multimedia, 18(12): 2457 - 2469, 2016. (download)
●

L. Li, H. Cai, Y. Zhang, W. Lin, A. C. Kot, X. Sun, “Sparse Representation Based Image Quality Index with Adaptive Sub-Dictionaries”, IEEE Transactions on Image Processing, 25(8): 3775 - 3786, 2016. (download)
●

F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, Q. Dai, “Learning blind quality evaluator for stereoscopic images using joint sparse representation”, IEEE Trans. Multimedia, 18(10): 2104-2114, 2016. (download)
●

L. Li, D. Wu, J. Wu, H. Li, W. Lin, A. C. Kot, “Image Sharpness Assessment by Sparse Representation”, IEEE Trans. Multimedia, 18(6): 1085-1097, 2016. (download)
●

F. Shao, W. Tian, W. Lin, G. Jiang, Q. Dai, “Towards a blind deep quality evaluator for stereoscopic images based on monocular and binocular interactions”, IEEE Transactions on Image Processing, 25(5): 2059 - 2074, 2016. (download)
●

F. Shao, W. Lin, G. Jiang, Q. Dai, “Models of Monocular and Binocular Visual Perception in Quality Assessment of Stereoscopic Images”, IEEE Transactions on Computational Imaging, 2(2): 123-135, 2016. (download)
●

K. Gu, S. Wang, G. Zhai, W. Lin, X. Yang, W. Zhang, “Analysis of Distortion Distribution for Pooling in Image Quality Prediction”, IEEE Trans. on Broadcasting, 62(2): 446-456, 2016. (download)
●
J. Wu, W. Lin, G. Shi, Y. Zhang, “Visual Orientation Selectivity based Structure Description”, IEEE Transactions on Image Processing, accepted, 2015. (download)
●

F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, Q. Dai, “Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties”, IEEE Transactions on Image Processing, VOL. 24, NO. 10, pp. 2971 – 2983, 2015. (download)
●

L. Dong, Y. Fang, W. Lin, H. S. Seah, “Perceptual Quality Assessment for 3D Triangular Mesh based on Curvature”, IEEE Trans. Multimedia, 17(12):2174 - 2184, 2015. (download)
●

F. Shao, W. Lin, S. Wang, G. Jiang, M. Yu, Q. Dai, “Learning receptive fields and quality lookups for blind quality assessment of stereoscopic images”, IEEE Transactions on Cybernetics, 2015. (download)
●

J. Wu, W. Lin, G. Shi, L. Li, “Orientation Selectivity based Visual Pattern for Reduced-Reference Image Quality Assessment”, Information Sciences, 351:18-29, 2016. (download)
●

Q. Li, W. Lin, Y. Fang, “No-reference quality assessment for multiply-distorted images in gradient domain”, IEEE SIGNAL PROCESSING LETTERS, 23(4):541-545, 2016. (download) (Source Codes)
●

F. Zhang, W. Jiang, F. Autrusseau, W. Lin, “Exploring V1 by modeling the perceptual quality of images”, Journal of Vision, VOL. 14, NO. 1, Article 26, 2014 (download)
●

F. Shao, S. Wang, W. Lin, G. Jiang, M. Yu, “Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook”, IEEE Trans.on Broadcasting, accepted. (download)
●

L. Li, W. Lin, X. Wang, G. Yang, K. Bahrami, A. C. Kot, “No-Reference Image Blur Assessment Based on Discrete Orthogonal Moments”, IEEE Transactions on Cybernetics, accepted. (download)
●

K. Gu, G. Zhai, X. Yang, W. Lin, W. Zhang, “No-Reference Image Sharpness Assessment in Autoregressive Parameter Space”, IEEE Transactions on Image Processing, accepted, 2015. (download)
●

Y. Fang, W. Lin, S. Winkler, “Review of Existing QoE Methodologies”, Multimedia Quality of Experience (QoE): Current Status and Future Requirements, eds. C. W. Chen, et al., John Wiley & Sons, 2015. (download)
●

F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, “Using Binocular Feature Rivalry for Blind Quality Assessment of Stereoscopic Images”, IEEE SIGNAL PROCESSING LETTERS,VOL. 22, NO. 10, pp. 1548 - 1551, 2015. (download)
●

L. Li, Y. Zhou, J. Wu, W. Lin, H. Li, “GridSAR: Grid strength and regularity for robust evaluation of blocking artifacts in JPEG images”, Journal of Visual Communication and Image Representation, accepted. (download)
●

S. Wang, K. Ma, H. Yeganeh, Z. Wang, W. Lin, “A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images”, IEEE SIGNAL PROCESSING LETTERS, 2015, accepted (download)
●

L. Li, Y. Zhou, W. Lin, J. Wu, X. Zhang, B. Chen, “No-reference quality assessment of deblocked images”, Neurocomputing, accepted (download)
●

Y. Fang, K. Ma, Z. Wang, W. Lin, Z. Fang, G. Zhai, “No-Reference Quality Assessment for Contrast-Distorted Images Based on Natural Scene Statistics”, IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 7, pp. 838 - 842, 2014, (download) (Source Codes)
●

C-C Hsu, C-W Lin, Y. Fang, W. Lin, “Objective Quality Assessment for Image Retargeting Based on Perceptual Geometric Distortion and Information Loss”, IEEE Journal of Selected Topics in Signal Processing, VOL. 8, NO. 3, pp. 377-389, 2014, (download)
●

J. Wu, W. Lin, G. Shi, “Image Quality Assessment with Degradation on Spatial Structure”, IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 4, pp. 437 - 440, 2014, (download)
●

L. Li, W. Lin, H. Zhu, “Learning Structural Regularity for Image Blockiness Evaluation”, IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 4, pp. 918 - 922, 2014, (download)
●

F. Zhang, W. Lin, Z. Chen, K. N. Ngan, “Additive Log-logistic Model for Networked Video Quality Assessment”, IEEE Transaction on Image Processing, VOL. 22, NO. 4, pp. 1536 - 1547, 2013 (download)
●
T-J Liu, W. Lin, and C.-C. Jay Kuo, “Image Quality Assessment Using Multi-Metric Fusion (MMF)”, IEEE Transaction on Image Processing, VOL. 22, NO. 5, pp. 1793-1807, 2013 (download)
The pioneer research as the foundation of the de-facto standard for video quality assessment in video streaming industry, VMAF (Video Multimethod Assessment Fusion, with open-source code released by Netflix & leading to the prestigious 72nd Annual Technology and Engineering Emmy Award for Development of Open Perceptual Metrics for Video Encoding Optimization in 2020).
●

T-J Liu, Y-C Lin, W. Lin, and C.-C. Jay Kuo, “Visual Quality Assessment: Recent Developments, Coding Applications and Future Trends”, APSIPA Transactions on Signal and Information Processing, Vol. 2, e4 (20 pages), 2013, (download)
●

J. Wu, W. Lin, G. Shi, A. Liu, “Reduced-Reference Image Quality Assessment with Visual Information Fidelity”, IEEE Trans. Multimedia, Vol. 15(7), pp. 1700-1704, 2013, (download)
●

L. Ma, C. Deng, K. N. Ngan, and W. Lin, “Recent Advances and Challenges of Visual Signal Quality Assessment”, China Communications, Vol. 10(5), pp. 62 - 78, 2013, (download)
●

F. Shao, W. Lin, S. Gu, G. Jiang, T. Srikanthan, “Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics”, IEEE Transaction on Image Processing, Vol. 22, No. 5, pp. 1940-1953, 2013 (download)
●

M. Narwaria, and W. Lin, "SVD-Based Quality Metric for Image and Video Using Machine Learning”, IEEE Trans. on Systems, Man, and Cybernetics--Part B, Vol. 42(2), pp. 347 - 364, 2012 (download) (Source Codes)
An old bottle for new wine: SVD for IQA
●

A. Liu, W. Lin, M. Narwaria, “Image Quality Assessment Based on Gradient Similarity”, IEEE Transaction on Image Processing, Vol. 21(4), pp. 1500 - 1512, 2012 (download) (Source Codes)
The research defines the gradient similarity (GS) for better IQA, remedying a major drawback in prior work.
●

M. Narwaria, W. Lin, I. McLoughlin, S. Emmanue, L. T. Chia, “Fourier Transform Based Scalable Image Quality Measure”, IEEE Trans. on Image Processing, Vol. 21(8), pp. 3364 – 3377, 2012 (download) (Source Codes)
The use of phase and magnitude of FT in scalable visual quality evaluation
●

Q. Xu, Y. Yao, T. Jiang, Q. Huang, W. Lin, B. Yan, “HodgeRank on Random Graphs for Subjective Video Quality Assessment”, IEEE Trans. Multimedia, Vol. 14(3), pp.844 – 857, 2012 (download)
To achieve efficient subjective assessment, without jeopardizing the accuracy
●

J. Wu, W. Lin, G. Shi, A. Liu, “Perceptual Quality Metric with Internal Generative Mechanism”, IEEE Trans. on Image Processing, VOL. 22, NO. 1, pp. 43-54, 2013 (download) (Source Codes)
Our eyes distinguishes orderly and disorderly visual signals, so does our quality metric.
●

M. Narwaria, W. Lin and A. Liu, “Low-Complexity VQA Using Temporal Quality Variations”, IEEE Trans. on Multimedia, Vol. 14(3), pp. 525-535, 2012 (download) (Source Codes)
Quality changes over time matter more.
●

G. Zhai , X. Wu, X. Yang, W. Lin, W. Zhang, “A Psychovisual Quality Metric in Free Energy Principle”, IEEE Transactions on Image Processing, Vol. 21(1), pp. 41 – 52, 2012, (download)
A fresh angle to look at picture quality prediction issues
●

W. Lin, C.-C. Jay Kuo, “Perceptual Visual Quality Metrics: A Survey”, J. of Visual Communication and Image Representation, Vol 22(4), pp. 297-312, May 2011 (download)
A comprehensive review of visual quality evaluation: our views
The Most Cited Journal of Visual Communication and Image Representation Article, 2011-2021
●

Q. Xu, Y. Yao, T. Jiang, Q. Huang, B. Yan, W. Lin, “Random Partial Paired Comparison for Subjective Video Quality Assessment via HodgeRank”, Long Paper, ACM Multimedia 2011. (download)
●

M. Narwaria, W. Lin, E. Cetin, “Scalable Image Quality Assessment with 2D Mel-cepstrum and Machine Learning Approach”, Pattern Recognition, vol. 45, no. 1, pp. 299-313, 2011 ( download)
●

M. Narwaria, and W. Lin, "Objective Image Quality Assessment Based on Support Vector Regression", IEEE Transactions on Neural Networks, Vol.21(3), pp.515-519, March 2010 (download).
This was among the first attempts to introduce and demonstrate machine learning as a new way of IQA. It avoids the unrealistic assumptions otherwise in the existing feature pooling frameworks.
●

G. Zhai, J. Cai, W. Lin, X. Yang, W. Zhang, M. Etoh, “Cross-dimensional Perceptual Quality Assessment for Low Bitrate Videos”, IEEE TRANSACTIONS ON MULTIMEDIA, vol.10 (7), pp. 1316-1324, Nov 2008.( download)
The work demonstrates that perceptual visual quality evaluation is a multidimensional problem (content, codec, image size, framerate, bitrate, and so on).
●

W. Lin, Gauging Image and Video Quality in Industrial Applications, in Advances of Computational Intelligence in Industrial Systems, eds. Y. Liu, et al, Springer-Verlag, Heidelberg, 2008, (download)
Although technology development of visual quality evaluation was still in its infancy, some deployment had been made in industries.
●

F. Pan, X. Lin, S. Rahardja, E. Ong and W. Lin, “Using edge direction information for measuring blocking artifacts of images”, Multidimensional Systems and Signal Processing, Vol 18 (4), pp. 297 – 308, December 2007, (download)
●

W. Lin, L. Dong, P. Xue, “Visual Distortion Gauge Based on Discrimination of Noticeable Contrast Changes”, IEEE Trans. Circuits and Systems for Video Technology, vol.15(7), pp. 900- 909, July, 2005, (download)
For long, an unreasonable assumption (and restriction) had been made in IQA that test images are all distorted.
Firstly, this research enables evaluation for enhanced images/videos, thus removing the said restriction.
Secondly, the proposed measure gracefully reduces to traditional mean-absolute-error (MAE), wherever JND approaches constant and contrast-changes are less differentiated; it allows easier user acceptance and comparison/benchmarking with traditional measures (like MAE, MSE or PSNR).
●

E. Ong, X. Yang, W. Lin, Z. Lu, S. Yao, X. Lin, S. Rahardja and C. Boon, “Perceptual Quality and Objective Quality Measurements of Compressed Videos”, Journal of Visual Communication and Image Representation, vol.17(4), pp.717-737, August 2006, (download)
●

E. Ong, W. Lin, Z. Lu, S. Yao and M. Etoh, “Visual Distortion Assessment with Emphasis on Spatially Transitional Regions”, IEEE Trans. Circuits and Systems for Video Technology, Vol. 14(4), PP.559 – 566, April 2004, (download)
●

F. Pan, X. Lin. S. Rahardja. W. Lin, E. Ong, S. Yao, Z. Lu and X. Yang, “A locally-adaptive algorithm for measuring blocking artifacts in images and videos”, Signal Processing: Image Communication, Vol 19(6), pp. 499-506, 2004, (download)

Authored book

L. Xu, W. Lin, and C.-C. Jay Kuo, Visual Quality Assessment by Machine Learning, Springer, 2015.

Edited book

C. Deng, L. Ma, W. Lin, and K. N. Ngan (eds.), Visual Signal Quality Assessment – Quality of Experience (QoE), Springer, 2014.

Fine-Grained (FG) IQA
A long-ignored problem in IQA: the coarse-grained (i.e., when quality variation is sufficiently big, as the setting of most IQA databases up to date) statistical results evaluated on existing databases mask the fine-grained differentiation...

X. Zhang, W. Lin, Q. Huang, “Fine-Grained Image Quality Assessment: A Revisit and Further Thinking”, IEEE Trans. on Circuits and Systems for Video Technology, 2021, accepted. (download)
X. Zhang, W. Lin, S. Wang, J. Liu, S. Ma, W. Gao, “Fine-Grained Quality Assessment for Compressed Images”, IEEE Transactions on Image Processing, 28(3): 1163 - 1175, 2019. (download) (database)
The first attempt for fine-grained IQA for same bitrate with different image coders: a more meaningful situation for image compression.

For Partially-Artificial (-Generated) Images (PAIs) ( link ) ( more info )

With recent rapid technological advancement of visual computing and networking, there have been increasing applications & services making use of computer partially generated images, or PAIs: screen content images (SCIs), retargeted images (RTIs), images/videos as a result of DIBR (depth-image-based rendering), style transfer, super-resolution (SR), computational photography (stitched panoramic images (SPIs), HDR (high dynamic range)), augmented images for machine-training, etc.

IQA for many types of PAIs is with no-reference (no full-reference) by nature, but we may turn it into a full-reference one in some cases, e.g., RTIs.

D. Mumtaz, Sadbhawna, V. Jakhetiya, B. N. Subudhi, W. Lin, "Non-Subsampled Contourlet Transform and Ground-truth Score Generation based Quality Assessment for DIBR-Synthesized Views”, IEEE Trans. on Multimedia, 2024, accepted. (download)
K. Zhang, T. Zhao, W. Chen, Y. Niu, J. Hu, W. Lin, “Perception-Driven Similarity-Clarity Tradeoff for Image Super-Resolution Quality Assessment”, IEEE Trans. on Circuits and Systems for Video Technology, accepted 2024, accepted. (download)
X. Zhang, N. Cai, H. Zhang, Y. Zhang, J. Di, W. Lin, “AFD-former: A Hybrid Transformer with Asymmetric Flow Division for Synthesized View Quality Enhancement”, IEEE Trans. on Circuits and Systems for Video Technology, 33(8): 3786-3798, 2023. (download)
H, Zhang, D. Zheng, Y. Zhang, J. Cao, W. Lin, W-K Ling, "Quality Assessment for DIBR-synthesized Views based on Wavelet Transform and Gradient Magnitude Similarity", IEEE Trans. on Multimedia,in press.
Q. Jiang, Z. Liu, K. Gu, F. Shao, X. Zhang, H. Liu, W. Lin, “Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and An Objective Metric”, IEEE Trans Image Process., 31: 2279-2294, 2022 (download) (dataset) (codes)
No-refernce assessment based on KLT + a dataset with 9 additional questions answered by subjects
Z. Peng, Q. Jiang, F. Shao, W. Gao, W. Lin,“LGGD+: Image Retargeting Quality Assessment by Measuring Local and Global Geometric Distortions”, IEEE Trans. on Circuits and Systems for Video Technology, 32(6): 3422-3437, 2022 (download)
V. Jakhetiya, S. Thakur, S. Badri, W. Lin, S. Guntuku, "Perceptually Unimportant Information Reduction and Cosine Similarity-Based Quality Assessment of 3D-Synthesized Images", IEEE Transactions on Image Processing, 31(2): 2027-2039, 2022 (download)
Q. Jiang, Z. Peng, F. Shao, K. Gu, Y. Zhang, W. Zhang, W. Lin, “StereoARS: Quality Evaluation for Stereoscopic Image Retargeting With Binocular Inconsistency Detection”, IEEE Trans. on Broadcasting, 68(1): 43 – 57, 2022. (download)
K. Gu, J. Qiao, S. Lee, H. Liu, W. Lin, P. Le Callet, “Multiscale Natural Scene Statistical Analysis for No-Reference Quality Evaluation of DIBR-Synthesized Views”, IEEE Transactions on Broadcasting, 66(1): 127-139, 2020. (download)
V. Jakhetiya. K. Gu, T. Singhal, S. C. Guntuku, Z. Xia, W. Lin, "A Highly Efficient Blind Image Quality Assessment Metric of 3D-Synthesized Images using Outlier Detection", IEEE Transactions on Industrial Informatics, 15(7): 4120-4128, 2019. (download )
Q. Jiang, F. Shao, W. Lin, G. Jiang, “BLIQUE-TMI: Blind Quality Evaluator for Tone-Mapped Images Based on Local and Global Feature Analyses”, IEEE Transactions on Circuits and Systems for Video Technology, 29(2): 323 - 335, 2019. (download )
For tone-mapped images (TMIs) from HDR ones: local & global loss to be evaluated; the major contribution is the former via sparse coding: local activity is derived from sparse coefficients with the TMI and its 8 darkened/brightened versions, to reflect the extent of exposure of the TMI, while primary structure is evaluated by the correponding dictionary atoms.
S. Wang, K. Gu, X. Zhang, W. Lin, S. Ma, W. Gao, “Reduced-Reference Quality Assessment of Screen Content Images”, IEEE Transactions on Circuits and Systems for Video Technology, 28(1): 1-14, 2018. (download)
Y. Zhang, W. Lin, Q. Li, W. Cheng, X. Zhang, "Multiple-Level Feature-Based Measure for Retargeted Image Quality", IEEE Trans. Image Processing 27(1): 451-463, 2018 (download )
Apart from Aspect Ratio Similarity (ARS) backward registration, more object and shape deformation is considered for RTI quality assessment.
K. Gu, J. Qiao, X. Min, G. Yue, W. Lin, D. Thalmann, “Evaluating Quality of Screen Content Images Via Structural Variation Analysis”, IEEE Transactions on Visualization and Computer Graphics, 24(10): 2689 - 2701, 2018. (download )
F. Shao, Q. Yuan, W. Lin, G. Jiang, “No-reference view synthesis quality prediction for 3D videos based on color-depth interactions”, IEEE Transactions on Multimedia, 20(3): 659-674, 2018. (download)
View synthesis distortion due to compression is predicted for 3D videos: the problem is decomposed as distortion due to compress color/texture (with original depth), distortion due to compress depth (with original color/texture), and color-depth interactions.
Y. Fang, J. Ya, L. Li, J. Wu, W. Lin, “No Reference Quality Assessment for Screen Content Images with Both Local and Global Feature Representation”, IEEE Transactions on Image Processing, 27(4): 1600 - 1610, 2018. (download)
S. Wang, K. Gu, K. Zeng, Z. Wang, W. Lin, “Objective Quality Assessment and Perceptual Compression of Screen Content Images”, IEEE Computer Graphics and Applications, 38(1): 47 – 58, 2018. ( download )
Soft thresholding to account for textual and pictorial regions in SCIs.
L. Li, Y. Zhou, K. Gu, W. Lin, S. Wang, “Quality Assessment of DIBR-Synthesized Images by Measuring Local Geometric Distortions and Global Sharpness”, IEEE TRANSACTIONS ON MULTIMEDIA, 20(4):914 – 926, 2018. (download)
3 steps: viewing-distance adaptive image resizing (into multi-scale images, as the base of assessment), disoccluded region detection & evaluation, and global sharpness evaluation after injection of fixed amount of noise to the synthesized image (which cannot be further blurred when is not sharp); the proposed scheme also boosts the performance of existing quality metrics, since it overcomes the common oversight among them.
K. Gu, V. Jakhetiya, J. Qiao, X. Li, W. Lin, D. Thalmann, “Model-Based Referenceless Quality Metric of 3D Synthesized Images Using Local Image Description”, IEEE Transactions on Image Processing, 27(1): 394-405, 2018. (download)
V. Jakhetiya, K. Gu, W. Lin, Q. Li, S. P. Jaiswal, “A Prediction Based Model for Quality Assessment of Screen Content and 3D synthesized Images”, IEEE Transactions on Industrial Informatics, 14(2):652 – 660, 2018. (download)
Q. Jiang, F. Shao, W. Lin, G. Jiang, “Learning Sparse Representation for Objective Image Retargeting Quality Assessment”, IEEE Transactions on Cybernetics, 48(4): 1276–1289, 2018. (download)
Retargeted image quality assessment with sparse correspondence, joint dictionary learning & sparse coding.
K. Gu, J. Zhou, J. F. Qiao, G. T. Zhai, W. Lin, A. C. Bovik, “No-reference quality assessment of screen content pictures”, IEEE Transactions on Image Processing, 26(8): 4005-4018, 2017. (download )
Feature extraction: image complexity (entropy) description, screen content statistics, global brightness (histogram) of image and its enhanced version, sharpness and corners; Training: applying 11 types of distortions to corrupt 800 SCIs to create 100,000 distorted images as training samples, labeled by an NR (blind) SCI metric.
X. Min, K. Ma, K. Gu, G. Zhai, Z. Wang, W. Lin, “Unified Blind Quality Assessment of Compressed Natural, Graphic, and Screen Content Images”, IEEE Transactions on Image Processing, 26(11): 5462 - 5474, 2017. (download)
Use of variance of local variance (VOLV) to differentiate natural, graphic & screen content images; in fact, screen content images can be viewed as a general form of natural ones (when there is no graphic part) or natural ones (when there is no natural part).

S. Wang, K. Gu, X. Zhang, W. Lin, L. Zhang, S. Ma, W. Gao, “Subjective and Objective Quality Assessment of Compressed Screen Content Images”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 6(4): 532 - 543, 2016. (download)
Features from wavelet domain: amplitude spectrum of an SCI in each subband, variance of coefficients, & entropy to reflect info content; learning for feature fusion via training with an FR-IQA meric instead of subjective data; construct a new database for SCI compression with HEVC: HEVC 4:4:4 and HEVC 4:2:0 compression artifacts, more content types, & high-resolution imagesm, as an additional database for validation of the proposed reduced-reference model.
K. Gu, S. Wang, G. Zhai, S. Ma, X. Yang, W. Lin, W. Zhang, W. Gao, “Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure”, IEEE Trans. Multimedia, 18(3): 432-443, 2016. (download)
Y. Zhang, Y. Fang, W. Lin, X. Zhang, L. Li, “Backward Registration Based Aspect Ratio Similarity (ARS) for Image Retargeting Quality Assessment”, IEEE Transactions on Image Processing, 25(9):4286-4297, 2016. (download) (Source Codes)
For benchmarking different image retargeting algorithms, the reference (original image) is available. The proposed dense correspondence estimation enables pixel backward registration from a retargeted image (RTI) to the original image, so that the assessment can make full use of the reference, i.e., the process is turned into a full-reference one.
H. Yang, Y. Fang, W. Lin, “Perceptual Quality Assessment of Screen Content Images”, IEEE Trans on Image Processing, 24(11): 4408 - 4421, 2015. (download) (database)
Segmenting textual and pictorial regions in SCIs to better facilitate quality assessment.
K. Gu, S. Wang, H. Yang, W. Lin, G. Zhai, X. Yang, W. Zhang, “Saliency-Guided Quality Assessment of Screen Content Images”, IEEE Transactions on Multimedia, 18(6): 1098 - 1110, 2016. (download)
K. Gu, G. Zhai, W. Lin, X. Yang, W. Zhang, “Learning a Blind Quality Evaluation Engine of Screen Content Images”, Neurocomputing, 196: 140 - 149, 2016 (download)
H. Yang, Y. Fang, Y. Yuan, W. Lin, “Subjective quality evaluation of compressed digital compound images”, Journal of Visual Communication and Image Representation, 26: 105-114, 2015. (download)
Screen images are increasingly important, so is the evaluation for them.
A. Liu, W. Lin, H. Chen, P. Zhang, “Image Retargeting Quality Assessment Based on Support Vector Regression”, Signal Processing: Image Communication, VOL. 39, Part B, pp. 444–456, 2015. (download)

Y. Fang, K. Zeng, Z. Wang, W. Lin, Z. Fang, C. Lin, “Objective Quality Assessment for Image Retargeting Based on Structural Similarity”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, VOL. 4, NO. 1, pp. 95 - 105, 2014, (download)
L. Ma, W. Lin, C. Deng, and K. N Ngan, “Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics”, IEEE Journal of Selected Topics in Signal Processing, Vol. 6(6), pp. 626 - 639, 2012 (download) (database

Evaluation of Other Signals

W. Chen, K. Gu, W. Lin, F. Yuan, E. Cheng, “Statistical and Structural Information Backed Full-Reference Quality Measure of Compressed Sonar Images”, IEEE Transactions on Circuits and Systems for Video Technology, 30(2): 334-348, 2020. (download) ( database )
An initial attempt for sonar image quality evaluation

●

G. Ghinea, C. Timmerer, W. Lin, S. Gulliver, “Mulsemedia: State-of-the- Art, Perspectives and Challenges”, ACM Transactions on Multimedia Computing Communications and Applications, Vol. 11(1s), Article 17, 2014 (download)
Humans have 5 senses; what can we expect from a machine?
A Notable Article in Computing, on ACM Computing Reviewers' 19th Annual Best of Computing List, 2014.
●

M. Narwaria, W. Lin, I. McLoughlin, S. Emmanue, L. T. Chia, “Nonintrusive Quality Assessment of Noise Suppressed Speech with Mel-Filtered Energies and Support Vector Regression”, IEEE Trans. on Audio, Speech and Language Processing, Vol. 20(4), pp. 1217 - 1232, 2012 (download)
A new attempt for speech quality evaluation: the machine-learning approach

Editorials

W. Lin, T. Ebrahimi, P. C. Loizou, S. Möller, A. R. Reibman, “Introduction to the Special Issue on New Subjective and Objective Methodologies for Audio and Visual Signal Processing”, Editorial, IEEE Journal of Selected Topics in Signal Processing, Vol. 6(6), pp. 614-615, 2012 (download)
W. Zeng and W. Lin, “QoE Modeling and Applications for Multimedia Systems” ZTE Communications, Vol. 11(1), 2013 (download)
T. Dagiuklas, W. Lin and A. Ksentini, “QoE Aware Optimization in Mobile Networks”, IEEE COMSOC MMTC E-Letter, Vol. 8, No. 2, March 2013.
T. Daguiklas, L. Atzori, P. Chatzimisios, C. Chen, W. Lin, “Special issue on QoE in 2D/3D video systems”, Journal of Visual Communication and Image Representation, VOL. 25, NO. 3, pp. 523-534, 2014.
G. Ghinea, C. Timmerer, W. Lin, S. Gulliver, “Special issue on Multiple Sensorial (MulSeMedia) Multi-modal Media: Advances and Applications”, ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP), Vol. 11(1s), 2014 (download).

Just-Noticeable Difference (JND) Formulation

To be noticeable, or not to be noticeable: that is the question.

Q. Jiang, Z. Liu, S. Wang, F. Shao, W. Lin,, “Towards Top-Down Just Noticeable Difference Estimation of Natural Images”, IEEE Transactions on Image Processing, 31(5): 3697-3712, 2022. (download) (Source Codes)
Existing handcrafted JND estimators are all bottom-up: model different masking effects in either spatial or frequency domain, and then fuse them as overall JND; this work is the first to estimate JND in a top-down manner: first predict a critical perceptual lossless (CPL) version of the original image; then calculate the difference between the original image and the CPL image as the JND map. In addition, this is the 1st time for JND to be estimated in KLT domain.
W. Lin and G. Ghinea, “Progress and Opportunities in Modelling Just-Noticeable Difference (JND) for Multimedia”, IEEE Transactions on Multimedia, 24: 3706-3721, 2022. (download)
Computational models for visual JND,which represent the majority of the related research so far, are first reviewed systematically, with both handcrafted modeling and machine-learning approaches. Furthermore, research attempts are then surveyed for JNDs for audio, smell, haptics and gustatory signals, as well as cross-modality/media efforts. Finally, possible future opportunities are substantially analysed.
V. Jakhetiya, W. Lin, S. Jaiswal, K. Gu, SC Guntuku, “Just Noticeable Difference for natural images using RMS contrast and feed-back mechanism”, Neurocomputing, 275: 366-376, 2018. (download)
The 1st attempt to approximaye spatial CSF (contrast sensitivity function) in pixel (spatial) domain.
J. Wu, L. Li, W. Dong, G. Shi, W. Lin, C.-C. Jay Kuo, “Enhanced Just Noticeable Difference Model for Images with Pattern Complexity”, IEEE Transactions on Image Processing, 26(6): 2682 - 2693, 2017 (download)
This research completed our yearslong pursuit for Pattern Masking for JND ...

S. Wang, L. Ma, Y. Fang, W. Lin, S. Ma, W. Gao, "Just Noticeable Difference Estimation for Screen Content Images", IEEE Trans. Image Processing 25(8): 3838-3851, 2016 (download)
Major differentiation in modeling JND for Screen Content Images (SCIs) against natural images: Parametric Edge Modeling
H. Bai, W. Lin, M. Zhang, A. Wang, Y. Zhao, “Multiple Description Video Coding Based on Human Visual System Characteristics”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 8, pp. 1390-1394, 2014 (download)
JND in multiple-description video coding
J. Wu, G. Shi, W. Lin, A. Liu, F. Qi, “Just Noticeable Difference Estimation For Images with Free-Energy Principle”, IEEE Trans. Multimedia, Vol. 15, No. 7, pp. 1705-1710, 2013, (download)
JND modeling with theoretical support from neuroscience
J. Wu, W. Lin, G. Shi, X. Wang, F. Li, “Pattern Masking Estimation in Image with Structural Uncertainty”, IEEE Transaction on Image Processing, VOL. 22, NO. 12, pp. 4892 - 4904, 2013 (download)
A extended JND modelor from the work above: with extraction of disorderly (unpredicted) portion of images for Pattern Masking Estimation.
Y. Gao, X. Xiu, J. Liang, W. Lin, “Fast Synthesized and Predicted Just Noticeable Distortion Maps for Perceptual Multiview Video Coding”, Journal of Visual Communications and Image Representation, VOL. 24, NO. 6, pp. 700-707, 2013 (download).
JND in multiview video coding
A. Liu, W. Lin, M. Paul, C. Deng, and F. Zhang, “Just Noticeable Difference for Images with Decomposition Model for Separating Edge and Textured Regions”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 20(11), pp. 1648-1652, 2010 (download) (Source Codes)
Differentiating different visual contents: the key for further JND model improvement
G. Zhai, W. Zhang, X. Yang, W. Lin, Y. Xu, “No-reference Noticeable Blockiness Estimation in Images”, Signal Processing: Image Communication, vol.23 (6), pp. 417-432, July 2008, (download)
JND is also estimated for just one kind of particular artifacts, e.g., blocking.
X. Zhang, W. Lin and P. Xue, “Just-Noticeable Difference Estimation With Pixels in Images”, Journal of Visual Communication and Image Representation, Vol 19(1), pp 30-41, 2008, (download)
Since JND is a characteristic of images, it should be able to convert from one domain to another. This work demonstartes the conversion from subband (transform) domain to pixel one.
Y. Jia, W. Lin and A. A. Kassim, “Estimating Just-Noticeable Distortion for Video”, IEEE Trans. Circuits and Systems for Video Technology, vol.16(7), pp. 820- 829, July, 2006, (download) (Source Codes)
Formulation of spatiotemporal CSF with considearion of eye movement
W. Lin, Computational Models for Just-noticeable Difference, Chapter 9 in Digital Video Image Quality and Perceptual Coding, eds. H. R. Wu and K. R. Rao, CRC Press, 2006, (download)
A general introduction & formulation on visual JND models-- a JND is the minimum amount of change for the difference to be detectable by humans, say 75% of the time. Next questions: what is the implication for 2nd JND, 3rd JND, ... ? How to derive them? JND for hearing, touching, smelling or even tasting?
X. Zhang, W. Lin, P. Xue, “Improved Estimation for Just-noticeable Visual Distortion”, Signal Processing, Vol. 85(4), pp.795-808, April 2005, (download) (Source Codes)
To model the basic image JND in transform domain, with a new formula for i) realistic luminance adaptation according to psychophysical findings in digitized (instead of analog) images, and ii) efficient block classification toward contrast-masking.
X. Yang, W. Lin, Z. Lu, E. Ong and S. Yao, “Just Noticeable Distortion Model and Its Applications in Video Coding”, Signal Processing: Image Communication, Vol. 20(7), pp. 662-680, August 2005, (download)
The well-accepted, perceptually-plausible NAMM (nonlinear additivity model for masking) formula for JND in spatial domain as generalization of all prior work, and demonstrated with its impact for video coding by “killing 3 birds with one stone”: higher perceived-quality, higher PSNR and lower computational-complexity.
X. Yang, W. Lin, Z. Lu, E. Ong, S. Yao, “Motion-compensated Residue Pre-processing in Video Coding Based on Just-noticeable-distortion Profile”, IEEE Trans. Circuits and Systems for Video Technology, vol.15(6), pp.742-750, June, 2005 (download) (Source Codes)
Much work has been done to optimize coders, and enabled by JND modeling, here we propose to optimize signals for better compressibility.

Visual Attention (VA) & Saliency Modeling

Since William James' time, the effort to understand and model human attention continues.

Y. Yun, W. Lin, “Towards a Complete and Detail-Preserved Salient Object Detection”, IEEE Trans. on Multimedia,26:4667-4680, 2024. (download) (Source Codes)
G. Li, Z. Liu, X. Zhang, W. Lin, “Lightweight Salient Object Detection in Optical Remote Sensing Images via Semantic Matching and Edge Alignment”, IEEE Transactions on Geoscience and Remote Sensing, 61: 1-11, 2023. (download)
S. Yang, W. Lin, G. Lin, Q. Jiang, Z. Liu, “Progressive Self-Guided Loss for Salient Object Detection”， IEEE Transactions on Image Processing, accepted. (download)
G. Li, Z. Liu, D. Zeng, W. Lin, H. Ling, “Adjacent Context Coordination Network for Salient Object Detection in Optical Remote Sensing Images”, IEEE Transactions on Cybernetics, 53(1): 526 – 538, 2023. (download)
G. Li, Z. Liu, W. Li n, H. Ling, “Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images”, IEEE Trans. on Geoscience and Remote Sensing, accepted (download)
S. Yang, G. Lin, Q. Jiang, W. Lin, “A Dilated Inception Network for Visual Saliency Prediction”, IEEE Transactions on Multimedia, 22 (8): 2163-2176, 2020. (download ) ( Source Codes )

An inception network with dilated kernels to address both multi-scales & computational efficiency in saliency prediction

Y. Fang, G. Ding, W. Wen, F. Yuan, Y. Yang, Z. Fang, W. Lin, "Salient Object Detection by Spatiotemporal and Semantic Features in Real-Time Video Processing Systems", IEEE Transactions on Industrial Electronics, 67(11): 9893-9903, 2020. (download)
J. Hou, S. Yang, W. Lin, “Object-level Attention for Aesthetic Rating Distribution Prediction”, ACM Multimedia, 2020. (download)
S. Yang, Q. Jiang, W. Lin, Y. Wang, “SGDNet: An End-to-End Saliency-Guided Deep Neural Network for No-Reference Image Quality Assessment”, ACM Multimedia, 2019. (download)

A. Nguyen, J. Kim, H. Oh, H. Kim, W. Lin, S. Lee, “Deep Visual Saliency on Stereoscopic Images”, IEEE Transactions on Image Processing, 28(4): 1939 – 1953, 2019. (download )
R. Cong, J. Lei, H. Fu, W. Lin, Q. Huang, X. Cao, C. Hou, “An Iterative Co-saliency Framework for RGBD Images”, IEEE TRANSACTIONS ON CYBERNETICS, 49(1): 233-246, 2019. (download)
To find common salient/attentional objects in multiple 3D (RGBD) images: effective iterations with saliency maps from single images as initialization, intra-image addition, and inter-image deletion
C-H Yeh, S-H Lo, W. Lin, "Visual Attention Based Pixel Dimming Technique for OLED Display on Mobile Devices"，IEEE Transactions on Industrial Electronics, 66(9): 7159-7167, 2019. (download)
T. Xi, W. Zhao, H. Wang, W. Lin, “Salient object detection with spatiotemporal background priors for video”, IEEE Transactions on Image Processing, 26(7): 3425-3436, 2017. (download)
F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, R. Fu, “An energy-constrained video retargeting approach for color-plus-depth 3D video”, IEEE/OSA Journal of Display Technology, 12(5): 491-499, 2016. (download)
J. Lei, B. Wang, Y. Fang, W. Lin, P. Le Callet, N. Ling, C. Hou, “A Universal Frameworkfor Salient Object Detection”, IEEE Trans. Multimedia, 18(9): 1783 - 1795, 2016. (download)
Y. Fang, J. Wang, Y. Yuan, J. Lei, W. Lin, P. Le Callet, “Saliency-Based Stereoscopic Image Retargeting”, Information Sciences, vol. 372, 347-358, 2016. (download)
K. Gu, G. Zhai, W. Lin, X. Yang, W. Zhang, “Visual Saliency Detection With Free Energy Theory”, IEEE SIGNAL PROCESSING LETTERS, 2015. (download)
Y. Fang, W. Lin, Z. Fang, Z. Chen, C-W Lin, C. Deng, “Visual Acuity Inspired Saliency Detection by Using Sparse Features”, Information Sciences, accepted (download)
F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting”, IEEE/OSA Journal of Display Technology. (download)
Y. Fang, J. Wang, M. Narwaria, P. Le Callet, W. Lin, “Saliency Detection for Stereoscopic Images”, IEEE Transaction on Image Processing,VOL. 23, NO. 6, pp. 2625 - 2636, 2014 (download) (Source Codes)
Y. Fang, W. Lin, Z. Chen, C-M Tsai, C-W Lin, “A Video Saliency Detection Model in Compressed Domain”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 1, pp. 27-38, 2014 (download)
Initial work for compressed-domain VA modeling for video.
H. Tian, Y. Fang, Y. Zhao, W. Lin, R. Ni, Z. Zhu, “Salient Region Detection by Fusing Bottom-up and Top-down Features Extracted from a Single Image”, IEEE Transaction on Image Processing, VOL. 23, NO. 10, pp. 4389-4398, 2014 (download) (Source Codes)
X. Bai, Y. Fang, W. Lin, L. Wang, B. Ju, “Saliency-Based Defect Detection in Industrial Images by Using Phase Spectrum”, IEEE Transactions on Industrial Informatics, VOL. 24, NO. 8, pp. 2135-2145, 2014 (download)
An example for possible industrial use of a VA model.
Y. Fang, Z. Wang, W. Lin, Z. Fang, “Video Saliency Incorporating Spatiotemporal Cues and Uncertainty Weighting”, IEEE Transaction on Image Processing, VOL. 23, NO. 9, pp. 3910-3921, 2014 (download) (Source Codes)
Adaptive uncertainty evaluation: decide which (spatial or temporal) contributes more to the final saliency.
L. Dong, W. Lin, Y. Fang, S. Wu, H. S. Seah, “Saliency detection in computer rendered images based on object-level contrast”, Journal of Visual Communication and Image Representation, VOL. 25, NO. 3, pp. 525-533, 2014 (download)
In comparison with natural images and video, perceptual computer graphics & animation are relatively less investigated so far, in spite of their importance.
Y.K.A. Low, X. Zou, Y. Fang, J.L. Wang, W. Lin, F.Y.C Boey, K.W. Ng, “β-Phase poly(vinylidene fluoride) films encouraged more homogeneous cell distribution and more significant deposition offibronectin towards the cell–material interface compared to α-phase poly(vinylidene fluoride) films”, Materials Science and Engineering C, Vol. 34, pp. 345-353, 2014 (download)
Visual saliency detection used in material engineering.
Y. Fang, Z. Chen, W. Lin, and C-W Lin, "Saliency Detection in the compressed domain for adaptive image retargeting", IEEE Trans. Image Processing, Vol. 21 (9), pp. 3888 - 3901, 2012 (download) (Codes)
The first work for compressed-domain VA modeling, which is significant since all images are stored and transmitted in compressed format; this enables visual saliency being directly determined without the need of decoding, for benefits in cost and power saving.
Y. Fang, W. Lin, B-S Lee, C. T. Lau, Z. Chen, C-W Lin, “Bottom-up Saliency Detection Model Based on Human Visual Sensitivity and Amplitude Spectrum”, IEEE Transactions on Multimedia, Vol. 14(1), pp. 187 - 198, 2012, (download) (Source Codes)
A visual attention model that considers human visual sensitivity variations due to foveation (i.e., the human visual sensitivity drops fast away from the attention center)
N. Imamoglu, W. Lin, and Y Fang, “A saliency detection model using low-level features based on wavelet transform”, IEEE Trans. Multimedia, Vol. 15(1), pp. 96 - 105, 2013 (download) (Source Codes)
Z. Lu, W. Lin, X. Yang, E. Ong and S. Yao, “Modeling Visual Attention's Modulatory Aftereffects on Visual Sensitivity and Quality Evaluation”, IEEE Trans. Image Processing, Vol.14(11), pp.1928 – 1942, Nov. 2005, (download)
Formulating VA’s modulatory effect on overall visual sensitivity in video, with a well-grounded basis to integrate bottom-up and top-down stimuli, according to relevant physiological and psychological knowledge.

Authored book

L. M. Zhang, W. Lin, Modeling Selective Visual Attention: Techniques and Applications, John Wiley & Sons, 2013.

Cross-modality Saliency Modeling

This book fills the gap of the basic VA theory and real-world applications (e.g., image retrieval, compression, retargeting, recognition, compressive sensing, IQA, and robotics), and provides a thorough and systematic coverage, backed by the intensive research by the authors and the teams they led during the 15 years before the publication.

W. Zhou, H. Zhang, W. Yan, W. Lin, “MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation”, IEEE Trans. on Circuits and Systems for Video Technology, 33(12):7096-7108, 2023. (download)
W. Gao, G. Liao, S. Ma, G. Li, Y. Liang, W. Lin, “Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection”, IEEE Trans. on Circuits and Systems for Video Technology, accepted. (download)
The first time for 1) a unified framework for saliency detection with both depth and thermal information; 2) cross-modal multi-stage feature fusion inspired by the effective visual color stage doctrine in the HVS.
G. Li, Z. Liu, M. Chen, Z. Bai, W. Lin, H. Ling, “Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection”, IEEE Transactions on Image Processing, 30(3): 3528 - 3542, 2021. ( download )
R. Cong, J. Lei, H. Fu, M. Cheng, W. Lin, Q. Huang, “Review of Visual Saliency Detection with Comprehensive Information”, IEEE Transactions on Circuits and Systems for Video Technology, 29(10): 2941-2959, 2019. (download)
A survey on RGBD saliency detection, co-saliency detection, and video saliency detection, using comprehensive info: depth, inter-image correspondence, temporal relationship, ...

Visual Signal Representation for Machine Intelligence (MI)

More and more images and videos are being used for various machine intelligence tasks without human interference in the AI Era...

Y. Cheng, Q. Guo, F. J. Xu, W. Feng, S-W Lin, W. Lin, Y. Liu, “Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack”, IEEE Transactions on Multimedia, 24: 3807 – 3822, 2022 (download)
To perform denoising/postprocessing (toward high-quality images) and enable model attack (identity/privacy preserving) simultaneously: by making an adversarial kernel similar to a denoise one
W. Sun, J. Jin, W. Lin, “Minimum Noticeable Difference based Adversarial Privacy Preserving Image Generation”, IEEE Trans. on Circuits and Systems for Video Technology, 33(3): 1069-1081, 2023 (download)
Would it be wonderful if our photos look good to friends in social media, and at the same time, human p rivacy is preserved in various machine tasks? This is a case with both humans and machines as users: make the change to exceed MI-JND but keep it within HVS-JND (if not possible, minimize the change beyond HVS-JND).
J. Jin, X. Zhang, X. Fu, H. Zhang, W. Lin, J. Lou, Y. Zhao, “Just Noticeable Difference for Deep Machine Vision”, IEEE Trans. on Circuits and Systems for Video Technology, 32(6): 3452-3461, 2022, (download)
This is the 1st model for JND derived for machine tasks (rather than human perception).
Z. Chen, K. Fan, S. Wang, L. Duan, W. Lin, A. C. Kot, “Toward Intelligent Sensing: Intermediate Deep Feature Compression”, IEEE Transactions on Image Processing, 29(12): 2230 - 2243, 2019. (download)
In the AI era, we can compress deep-learnt visual features (rather than whole visual content); this enables accurate feature extraction (before compression), edge computing and privacy preserving, as well as computation/battery saving at numerous client terminals.
Z. Chen, K. Fan, S. Wang, L. Duan, W. Lin, A. Kot, “Lossy Intermediate Deep Learning Feature Compression and Evaluation”, ACM Multimedia, 2019. (download) (Source Codes)

Perceptual Visual Coding & Processing

We are making every bit and operation count, and count for users.

C-H Yeh, S-H Lo, W. Lin, "Visual-Quality Guided Backlight Global Dimming for Video Display on Mobile Devices"，IEEE Transactions on Circuits and Systems for Video Technology, 29(11): 3393-3403, 2019. (download )
X. Zhang, S. Wang, K. Gu, W. Lin, S. Ma, W. Gao, "Just-Noticeable Difference Based Perceptual Optimization for JPEG Compression,” IEEE Signal Processing Letters, vol.24, no.1, pp.96-100, Jan. 2017 (download)
JND is used for RDO in video decoding. (emal for Source Codes)
V. Jakhetiya, W. Lin, S. P. Jaiswal, S. C. Guntuku, O. C. Au, “Maximum a Posterior and Perceptually Motivated Reconstruction Algorithm: A Generic Framework”, IEEE Trans. Multimedia, 19(1): 93 - 106, 2017 (download)
F. Shao, W. C. Lin, W. Lin, Q. Jiang, G. Jiang, “QoE-guided warping for stereoscopic image retargeting”, IEEE Transactions on Image Processing, 26(10): 4790– 4805, 2017. (download)
K. Gu, G. Zhai, W. Lin, M. Liu, “The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement”, IEEE Trans. Cybernetics, 46(1): 284-297, 2016 (download)
This work builds the 1st dedicated contrast-changed image database and automates visual contrast enhancement with the devised reduced-reference (RR) IQA; in addition, it demonstrates that an RR metric can be used as a no-reference (NR) one when only a specific image is dealt, e.g., in an enhancement process, since the RR remains as a constant while different enhanced versions are compared.
S. Guntuku, J. T. Zhou, S. Roy, W. Lin, I. W. Tsang, “Understanding Deep Representations Learned in Modeling User `Likes'”, IEEE Transactions on Image Processing, 25(9): 3762-3774, 2016. (download)
L. Xu, W. Lin, L. Ma, Y. Fang, K. Ngan, S. Li, Y. Yan, “Free-energy Principle Inspired Video Quality Metric and Its Use in Video Coding”, IEEE Trans. Multimedia, VOL. 18, NO. 4, pp. 590 - 602, 2016. (download)
F. Shao, W. Lin, G. Jiang, M. Yu, “Low-complexity depth coding by depth sensitivity aware rate-distortion optimization”, IEEE Trans. on Broadcasting, 62(1): 94-102, 2016 (download)
S. Wang, K. Gu, S. Ma, W. Lin, W. Gao, “Guided Image Contrast Enhancement Based on Retrieved Images in Cloud”, IEEE Trans. Multimedia, accepted, 2015 (download)
F. Shao, W. Lin, G. Jiang, M. Yu, Q. Dai, “Depth Map Coding for View Synthesis Based on Distortion Analyses”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, VOL. 4, NO. 1, pp. 106-117, 2014 (download)
L. Dong, Y. Fang, W. Lin, C. Deng, C. Zhu, H. S. Seah, “Exploiting entropy masking in perceptual graphic rendering”, Signal Processing: Image Communication, accepted, 2015. (download)
Unlike in natural images and video, we do not have the original visual signals as the reference in computer graphics, so perception is the only criterion for processing.
H. R. Wu, A. Reibman, W. Lin, F. Pereira, S. S. Hemami, “Perceptual Visual Signal Compression and Transmission”, PROCEEDINGS OF THE IEEE, VOL. 101, NO. 9, pp. 2025 - 2043, 2013 (download)
A survey for perceptual image/video coding and transmission.
F. Shao, G. Jiang, W. Lin, Y. Mei, Y. H. Dai, “Joint Bit Allocation and Rate Control for Coding Multi-view Video plus Depth based 3D Video”, IEEE Trans. Multimedia, Vol. 15(8), pp. 1843 - 1854, 2013 (download)
M. Paul, W. Lin, “Computer vision aided video coding”, in Advanced Video Communications Over Wireless Networks, C. Zhu and Y Li (eds.), CRC Press, 2012. (download)
F. Zhang, W. Liu, W. Lin, K. N. Ngan, “Spread Spectrum Image Watermarking Based on Perceptual Quality Metric”, IEEE Transaction on Image Processing, VOL. 20, NO. 11, pp. 3207 – 3218, Nov 2011, (download) (Source Codes)
S. Wu, S. Xie, W. Lin, “Blind Measurement of Image Blur for Vision-based Applications”, Multimedia Analysis, Processing and Communications, eds. W. Lin, et al., pp.185-215, Springer, 2011 (download) (Source Codes)
Visual quality evaluation can be used as pre-processing for computer vision based applications.
S. Wu, W Lin, S. Xie, Z. Lu, E. Ong, S. Yao, “Blind Blur Assessment for Vision-based Applications”, Journal of Visual Communication and Image Representation, Vol 20(4), pp 231-241, 2009 (download) (Source Codes)
W. Lin, Y. Gai. A. A. Kassim, “A Study on Perceptual Impact of Edge Sharpness in Images”, IEE Proc. Vision, Image & Signal Processing, vol. 153(2), pp. 215-223, April 2006, (download)
To which extent can we enhance image edge?
X. Yang, W. Lin, Z. Lu, X. Lin, S. Rahardja, E. Ong, S. Yao, “Rate Control for videophone using perceptual sensitivity cues”, IEEE Trans. Circuits and Systems for Video Technology, vol 15(4), pp.496-507, April, 2005 (download)

3D Visual Model Acquisition & Processing

K. Chan, F. Liu, G. Lin, C. S. Foo W. Lin, “R-Cyclic Diffuser: Reductive and Cyclic Latent Diffusion for 3D Clothed Human Digitalization”, CVPR 2024.
K. Chan, F. Liu, G. Lin, C. S. Foo, W. Lin, “Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction”, AAAI 2024. (download)
Z. Zhang, W. Sun, H. Wu, Y. Zhou, C. Li, Z. Chen, X. Min, G. Zhai, W. Lin, “GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment”, ACM Transactions on Multimedia Computing Communications and Applications, 2024, accepted. (download)
C. Lv, W. Lin, B. Zhao, "KSS-ICP: Point Cloud Registration based on Kendall Shape Space", IEEE Transactions on Image Processing, 32: 1681-1693, 2023. (download)
X Ding, Z Chen, W. Lin, Z Chen ,“Towards 3D Colored Mesh Saliency: Database and Benchmarks”, IEEE Trans. on Multimedia, 26:3580-3591, 2023 (download)
Y Zhang, M Wang, J Li, S Wang, S Ma, W. Lin, “Toward the Achievable Rate-Distortion Bound of VVC Intra Coding: A Beam Search-Based Joint Optimization Scheme”, IEEE Transactions on Image Processing, 32:6020-6031, 2023. (download)
Sadbhawna, V. Jakhetiya, B. N. Subudhi, S. P. Jaiswal, L. Li, W. Lin, “Context Region Identification based Quality Assessment of 3D Synthesized Views”, IEEE Transactions on Multimedia, accepted. (download)
C. Lv, W. Lin, B. Zhao, “Intrinsic and Isotropic Resampling for 3D Point Clouds”, IEEE Trans on Pattern Analysis and Machine Intelligence, accepted, 2022 (download) (Source Codes)
To provide geodesic measurement for a PC to improve local region detection & avoid redundant geodesic calculation, and then use a geometric update process to optimize the PC into an isotropic or adaptively-isotropic one
J. Xiong, H. Gao, M. Wang, H. Li, K. N. Ngan, W. Lin, “Efficient Geometry Surface Coding in V-PCC”, IEEE Transactions on Multimedia, in press. (download)
W. Gao, S. Fan, G. Li, W. Lin, “A Thorough Benchmark and A New Model for Light Field Saliency Detection”, IEEE Trans on Pattern Analysis and Machine Intelligence, in press. (download)
C. Lv, W. Lin, B. Zhao, "KSS-ICP: Point Cloud Registration based on Kendall Shape Space", IEEE Transactions on Image Processing, 32: 1681-1693, 2023. (download)
C. Lv, W. Lin, B. Zhao, “Voxel Structure-based Mesh Reconstruction from a 3D Point Cloud”, IEEE Transactions on Multimedia, accepted (download) (Source Codes)
PC To Mesh using intrinsic control: search structure, maintain geometric features and aim for being isotropic (with equilateral triangles) (More detail); intrinsic (non-Euclidean) geometric analysis is crucial for PC processing.
C. Lv, W. Lin, J. Zheng, “Adaptively Isotropic Remeshing based on Curvature Smoothed Field”, IEEE Transactions on Visualization and Computer Graphics, in press. (downloa)
K. Chan, G. Lin, H. Zhao, W. Lin, “S-PIFu: Integrating Parametric Human Models with PIFu for Single-view Clothed Human Reconstruction”, NeurIPS 2022. (downloa)
K. Chan, G. Lin, H. Zhao, W. Lin, “IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction”, ECCV, 2022. (downloa)
W. Lin, S. Lee, “Visual Saliency and Quality Evaluation for 3D Point Clouds and Meshes: An Overview”, APSIPA Trans. on Signal and Information Processing,11(1), e28, 2022 (download)
A survey with major perspectives from: 1) ultimate users (i.e., humans or machines); 2) modeling methodology (with handcrafted features or machine learning); 3) modeling scope (generic or utility-oriented models).
C. Lv, W. Lin, B. Zhao, "Approximate Intrinsic Voxel Structure for Point Cloud Simplification", IEEE Transactions on Image Processing, 30(9): 7241 – 7255, 2021 (download) (Source Codes)
PC simplification is a necessary step after data aquisition, with the following objectives: 1) point distribution uniformity in local regions; 2) geometric consistency with the original PC; 3) low computational cost; 4) flexibility toward user specifications.
J. Xiong, H. Gao, M. Wang, H. Li, W. Lin, “Occupancy Map Guided Fast Video based Dynamic Point Cloud Coding”, IEEE Transactions on Circuits and Systems for Video Technology, accepted. (download)
3 simple rules based on occupancy maps help to reduce nearly a half of computation with negligible loss of PC compression quality; these are compliant to the V-PCC standard.
B. Zhao, W. Lin, C. Lv, “Fine-Grained Patch Segmentation and Rasterization for 3D Point Cloud Attribute Compression”,IEEE Trans. on Circuits and Systems for Video Technology, 31(12): 4590-4602, 2021. (download)
W. Cheng, W. Lin, K. Chen, X. Zhang, “Cascaded Parallel Filtering for Memory Efficient Image-based Localization”, International Conference on Computer Vision (ICCV), 2019. (download)
Major contributions for image-based localization with SfM PCs: binary feature representation, significance-driven 2D-3D match disambiguation, and quality-aware pose estimation
X. Ding, W. Lin, Z. Chen, X. Zhang, “Point Cloud Saliency Detection by Local and Global Feature Fusion”, IEEE Transactions on Image Processing, 28(11): 5379–5393, 2019. (download)
2 contributions: 1) inclusion of local features and use of Random Walk Ranking for global rarity determination; 2) adaptive fusion of local distinctness and global rarity.
W. Cheng, K. Chen, W. Lin, M. Goesele, X. Zhang, Y. Zhang, "A Two-stage Outlier Filtering Framework for City-Scale Localization using 3D SfM Point Clouds", IEEE Transactions on Image Processing, 28(10): 4857 - 4869, 2019 (download)
Visibility (significance) and geometry based outlier filtering for match disambiguation in 3D SfM PCs
W. Cheng, W. Lin, X. Zhang, M. Goesele, M-T Sun, “A Data-driven Point Cloud Simplification Framework for City-scale Image-based Localization”, IEEE Transactions on Image Processing, 26(1): 262-275, 2017. (download)
Sufficiency of 2D-3D correspondences H is proposed for an arbitrary view in a PC, together with its computing model: a reliable ground for PC quality measurement, simplification, localization, and so on, as a result of consideration of the PC's info density in its formation. This work is based on Structure from Motion (SfM) PCs.
S. M. Prakhya, B. Liu, W. Lin, V. Jakhetiya, S. C. Guntuku, “B-SHOT: A Binary 3D Feature Descriptor for Fast Keypoint Matching on 3D Point Clouds”, Autonomous Robots, 41(7):1501–1520, 2017. (download) (Source Codes)
S. M Prakhya, J. Lin, V. Chandrasekhar, W. Lin, B. Liu, "3DHoPD: A Fast Low Dimensional 3D Descriptor", IEEE Robotics and Automation Letters, 2(3): 1472-1479, 2017. (download) (Source Codes)
A 3D feature descriptor with histograms of number of points and their 3D positions.
S. M. Prakhya, W. Lin, V. Chandrasekhar, B. Liu, J. Lin, “Low Bit-rate 3D Feature Descriptors for Depth Data from Kinect-style Sensors”, Signal Processing: Image Communication, 51: 40–49, 2017 (download)
S. M. Prakhya, B. Liu, W. Lin, “Detecting Keypoint Sets on 3D Point Clouds via Histogram of Normal Orientations”, Pattern Recognition Letters, 83 (Part 1): 42–48, 2016. (download)
A set of keypoints where there is high curvature, instead of single points at the maximum curvature that might sometimes arise due to noise, are used for more reliable detection of keypoints via Histogram of Normal Orientations in a 3D PC.
S. M. Prakhya, B. Liu, W. Lin, “B-SHOT: A binary feature descriptor for fast and efficient keypoint matching on 3D point clouds”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015 (download)
32 times less memory & 6 times faster when compared to SHOT.

Other Visual Signal Coding & Processing

Y Zhang, M Wang, J Li, S Wang, S Ma, W. Lin, “Toward the Achievable Rate-Distortion Bound of VVC Intra Coding: A Beam Search-Based Joint Optimization Scheme”, IEEE Transactions on Image Processing, 32:6020-6031, 2023 ( download )
H Wang, L Liao, J Xiao, W.Lin, M Wang, “Uplink-Assist Downlink Remote Sensing Image Compression via Historical Referencing”, IEEE Transactions on Geoscience and Remote Sensing, 61, 2023 (download)
J. Jin, X. Zhang, L. Meng, W. Lin, J. Liang, H. Zhang, Y. Zhao, "Auto-Weighted Layer Representation Based View Synthesis Distortion Estimation for 3-D Video Coding", IEEE Trans. on Multimedia, in press. ( download )
Z. Chen, K. Fan, S. Wang, L. Duan, W. Lin, A. Kot, “Lossy Intermediate Deep Learning Feature Compression and Evaluation”, ACM Multimedia, 2019. (download )
Z. Chen, K. Fan, S. Wang, L. Duan, W. Lin, A. C. Kot, “Toward Intelligent Sensing: Intermediate Deep Feature Compression”, IEEE Transactions on Image Processing, 29(12): 2230 - 2243, 2019. (download)
In the AI era, we can compress deep-learnt visual features (rather than whole visual content); this enables accurate feature extraction (before compression), edge computing and privacy preserving, as well as computation/battery saving at numerous client terminals.
X. Zhang, W. Lin, Y. Zhang, S. Wang, S. Ma, L. Duan, W. Gao, “Rate-Distortion Optimized Sparse Coding with Ordered Dictionary for Image Set Compression”, EEE Transactions on Circuits and Systems for Video Technology, 28(12): 3387-3397, 2018. (download) (emal for Source Codes)
Explore set redundance among similar images, by selecting a representative image to train a dictionary for each set.
K. Fan, R. Wang, W. Lin, L. Duan, W. Gao, “Signal-Independent Separable KLT by Offline Training for Video Coding”, IEEE Access, 7, pp. 33087 – 33093, 2019. (download)
G. Yue, C. Hou, J. Lei, Y. Fang, W. Lin, “Optimal region selection for stereoscopic video subtitle insertion”, IEEE Transactions on Circuits and Systems for Video Technology, 28(11): 3141 – 3153, 2018. (download)
X. Zhang, R. Xiong, W. Lin, Jian Zhang, Shiqi Wang, Siwei Ma, Wen Gao, ”Low-Rank Based Nonlocal Adaptive Loop Filter for High Efficiency Video Compression”, IEEE Trans. on Circuits and Systems for Video Technology, vol.27, no.10, pp.2177-2188, Oct. 2017 (download)
SVD is used for loop filtering in video decoding. (emal for Source Codes)
X. Zhang, S. Wang, Y. Zhang, W. Lin, S. Ma, W. Gao, "High-Efficiency Image Coding via Near-Optimal Filtering", IEEE Signal Processing Letters, vol. 24, no. 9, pp. 1403-1407, Sept. 2017. (download)
Illustrating what is the best result of optimal (and near-optimal) filtering for image compression.
X. Zhang, W. Lin, R. Xiong, X. Liu, S. Ma, W. Gao, “Low-Rank Decomposition Based Restoration of Compressed Images via Adaptive Noise Estimation”, IEEE Transactions on Image Processing, 25(9): 4158 - 4171, 2016 (download)
SVD is used for post-filtering of JPEG images.
H. Yang, S. Wu, C. Deng, W. Lin, “Scale and Orientation Invariant Text Segmentation for Born-Digital Compound Images”, IEEE Transactions on Cybernetics, VOL. 45, NO. 3, pp. 533 - 547, 2015 (download)
Screen images are increasingly important, so is the processing for them.
Y. Yuan, Y. Fang, W. Lin, “Visual Object Tracking by Structure Complexity Coefficients”, IEEE Trans. Multimedia, 17(8):1125 - 1136, 2015. (download)
Target appearance change is a major challenge in object tracking. In this work, structural and smooth regions are differentiated in similarity evaluation of target appearance, due to their inherent difference in appearance stability.
X. Zhang, R. Xiong, W. Lin, S. Ma, J. Liu, W. Gao, "Video Compression Artifact Reduction via Spatio-Temporal Multi-Hypothesis Prediction", IEEE Transactions on Image Processing, 24(12):6048-6061, 2015 (download)
The first use of quantization noise, nonlocal similarity & temporal AR models (as 3 hypotheses) to be fused in video decoding. (emal for Source Codes)
S. Wang, Z. Wang, S. E. M. Foo, N. S. Tan, Y. Yuan, W. Lin, Z. Zhang, K. W. Ng, “Culturing Fibroblasts in 3D Human Hair Keratin Hydrogels”, ACS Applied Materials & Interfaces, accepted (download)
Y. Yuan, S. Emmanuel, Y. Fang, W. Lin, “Visual Object Tracking based on Backward Model Validation”, IEEE Transactions on Circuits and Systems for Video Technology, VOL. 24, NO. 11, pp. 1898 - 1910, 2014 (download)
Occusion is a major challenge in object tracking: backward (i.e., using future frames) model validation helpful in differentiating occlusion and large appearance change.
L. Zhang, L. Wang, W. Lin, S. Yan, “Geometric Optimum Experimental Design for Collaborative Image Retrieval”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, pp. 346 - 359, 2014 (download)
J. Wu, W. Lin, G. Shi, J. Xiao, “Correlation based Universal Image/Video Coding Loss Recovery”, Journal of Visual Communication and Image Representation,VOL. 25, NO. 7, pp. 1507–1515, 2014, (download)
Using the correlation among coded pixels to reduce coding loss.
M. Paul, W. Lin, C. T. Lau, B-S Lee, “A Long Term Reference Frame for Hierarchical B-Picture based Video Coding”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 10, pp. 1729-1742, 2014 (download)
Since a natural video frame cannot be an ideal reference frame for coding in general, we generate a man-made one to serve the purpose.
W Liu, W Lin, “Gaussian Noise Level Estimation in SVD Domain for Images”, IEEE Transaction on Image Processing, VOL. 22, NO. 3, pp. 872 - 883, 2013 (download) (Source Codes)
SVD provides a good a basis for noise level estimation in images: signal contribution decreases rapidly in singular values, while noise contribution remain throughout singular values -- so, the "tail" of singular values tells about noise! This work adds Gaussian noise to an image to facilitate noise level detection in the original image, while for AIGC for visual signals, different leveles of Gaussian noise addition facilitate Diffusion models.
C-M Tsai, L-W Kang, C-W Lin, W. Lin, “Scene-Based Movie Summarization via Role-Community Networks”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, Vol. 23(11), pp. 1927 - 1940, 2013 (download)
M. Paul, W. Lin, C. T. Lau, B-S Lee, “Video Coding with Dynamic Background”, EURASIP Journal on Advances in Signal Processing, 11, 2013 (view)
Z. Gu, W. Lin, B-S Lee, C. T. Lau, M-T Sun, “Mode Dependent Templates and Scan Order for H.264/AVC Based Intra Lossless Coding”, IEEE Transaction on Image Processing, Vol. 21(9), pp. 4106 - 4116, 2012 (download)
How to engineer more zeros in bitstream?
L. Zhang, L. Wang, W. Lin, “Conjunctive Patches Subspace Learning with Side Information for Collaborative Image Retrieval”, IEEE Transactions on Image Processing, Vol. 21(8), pp. 3707 - 3720, 2012, (download)
Z. Gu, W. Lin, B.-S. Lee, C. T. Lau, “Low Complexity Video Coding Based on Two-dimensional Singular Value Decomposition (2D-SVD)”, IEEE Trans. on Image Processing, Vol. 21(2), pp. 674 - 687, 2012 (download)
2D-SVD as the transform for coding
C. Deng, W. Lin, B-S Lee, C. T. Lau, “Robust Image Coding Based upon Compressive Sensing”, IEEE Transactions on Multimedia, Vol. 14(2), pp. 278 - 290, 2012 (download)
C. Deng, W. Lin, and J. Cai, “Content-based Image Compression for Arbitrary-resolution Display Devices”, IEEE Trans. Multimedia, Vol. 14(4), pp. 1127 - 1139, 2012 (download)
We make a single bit stream equipped with decoding possibility of arbitrary image sizes.
L. Zhang, L. Wang, W. Lin, “Semi-supervised biased maximum margin analysis for interactive image retrieval”, IEEE Transactions on Image Processing, Vol. 21(4), pp. 2294 - 2308, 2012. (download)
Z. Gu, W. Lin, B-S Lee, C. T. Lau, “Rotated Orthogonal Transform (ROT) for Motion-Compensation Residual Coding” IEEE Transaction on Image Processing, VOL. 21, NO. 12, pp. 4770 – 4781, DECEMBER 2012 (download)
If we worry about the following two aspects of transforms used in video coding:
i) a traditional (and fixed) transform (like DCT or Wavelet) cannot be optimal for all video, and
ii) a adaptive trasform (like SVD, etc.) needs to specify the transform itself (this causes bits),
then this work is a solution: start with DCT (guaranteeing a better result than DCT), and rotate DCT basis fuctions adaptively for visual content (the rotation can be specified efficiently).
L. Zhang, L. Wang, W. Lin, “Generalized Biased Discriminant Analysis for Content-Based Image Retrieval”, IEEE Transactions on Systems, Man, and Cybernetics--Part C, Vol. 42(1), pp. 282 - 290, 2012, (download)
Our way to tackle the problems of the positive within-class scatter and the Gaussian distribution assumption for positive samples, in Discriminant Analysis
M. Liu, HS Seah, C. Zhu, W. Lin, F. Tian, “Reducing Location Map In Prediction-Based Difference Expansion For Reversible Image Data Embedding Signal Processing”, Signal Processing, Vol. 92, Issue 3, pp. 819-828, March 2012, (download)
A. Liu, W. Lin, M. Paul, F. Zhang, C. Deng, “Optimal Compression Plane for Efficient Video Coding”, IEEE Transaction on Image Processing, 20(10), pp.2788-2799, 2011, (download) (Source Codes)
"Out of the box" thinking: who says that a frame has to be in XY plane always?
M. Paul, W. Lin, C. T. Lau, B. –S. Lee, “Explore and Model Better I-frames for Video Coding”, IEEE Trans. Circuits and Systems for Video Technology,Vol. 21(9), pp. 1242 – 1254, 2011, (download) (Source Codes)
A "golden frame" we make for motion estimation and as an I frame, since a natural frame cannot do the two jobs well.
M. Paul, W. Lin, C. T. Lau, B. –S. Lee, “Direct Inter-Mode Selection for H.264 Video Coding using Phase Correlation,” IEEE Transaction on Image Processing, 20(2), pp. 461-473, Feb. 2011, (download)
C. Deng, W. Lin, B-S Lee, C. T. Lau, M-T Sun, "Performance Analysis, Parameter Selection and Extensions to H.264/AVC FRExt for Super-High Definition Video Coding”, Journal of Visual Communication and Image Representation, Vol.22(8), pp. 749-759, 2011, (download)
Does H.264 still work well for coding HD video? Yes, if you read this paper:-).
G. Zhai, X. Yang, W. Lin and W. Zhang, “Bayesian Error Concealment with DCT Pyramid for Images”, IEEE Trans. Circuits and Systems for Video Technology, Vol 20(9), pp. 1224-1232, Sept 2010, (download)
Choose the right level to start making a sensible decision...for image restoration
M. Shen, C. Wang, P. Xue and W. Lin, “Performance of Reconstruction-based Super-resolution with Regularization”, Journal of Visual Communications and Image Representation, Vol.21(7), pp.640-650, Oct. 2010, (download)
G. Zhai, W. Zhang, X. Yang, W. Lin and Y. Xu, “Unified Deblocking for DCT Compressed ”, J. of Electronic Imaging, Vol. 17(4), 2009, (download)
G. Zhai, W. Lin, J. Cai , X. Yang and W. Zhang, “Effective Quadtree based Block-shift Filtering for Deblocking and Deringing”, Journal of Visual Communication and Image Representation, vol.20(8), pp.595-607, Nov 2009, (download)
P. Li, W. Lin, X.Yang, “Analysis of H.264/AVC and an Associated Rate Control Scheme”, J. of Electronic Imaging, Vol. 17(4), 2009, (download)
G. Zhai, W. Zhang, X. K. Yang, W. Lin, and Y. Xu, “Efficient deblocking with coefficient regularization, shape adaptive filtering and quantization constraint”, IEEE Transactions on Multimedia, vol. 10(8), pp. 735-745, 2008, (download)
S. Wu, W Lin, and S. Xie, “Skin Heat Transfer Model of Facial Thermograms and Its Application in Face Recognition”, Pattern Recognition, Vol. 41(8), pp. 2718-2729, 2008, (download)
Blood perfusion rate reveals more about a person, so facilitates better face recognition.
G. Zhai, W. Zhang, X. Yang, W. Lin and Y. Xu, “Efficient Image Deblocking Based on Postfiltering in Shifted Windows”, IEEE Trans. Circuits and Systems for Video Technology. Vol. 18 (1), pp.122 – 126, Jan. 2008, (download)
C Wang, P. Xue, W. Lin, “Layered Image Resizing in Compression Domain”, Signal Processing: Image Communication, Vol.23 (1) , pp. 58-69, January 2008, (download)
G. Zhai, J. Cai , W. Lin, X. Yang, and W. Zhang, “Three Dimensional Scalable Video Adaptation via User-end Perceptual Quality Assessment”, IEEE Trans. Broadcasting, Vol. 54(3), pp. 1316-1324, 2008, (download)
E. Ong, W. Lin, “Video Object Segmentation”, in Encyclopedia of Information Communication Technology (ICT) , Ideal Group Inc., A. Cartelli and M. Palma (eds.), 2008,(download)
H. Gao, J. Tham, P. Xue and W. Lin, “Complexity Analysis of Morphological Area Openings and Closings with Set Union”, IET Image Processing, Vol. 2, No. 4, pp. 231-238, August 2008, (download)
W. Lin and L. Dong, “Adaptive Down-sampling to Improve Image Compression at Low Bit Rates”, IEEE Trans. Image Processing, Vol.15(9), pp. 2513-2521, Sept. 2006, (download) (Source Codes)
When bits become too few to code all pixels, it is better to downsample while encoding and then restore them at the receiving end. A way to solve big visual data problem...
C. Wang, P. Xue, W. Lin, W. Zhang, S. Yu, “Fast Edge-Preserved Postprocessing for Compressed Images”, IEEE Trans. Circuits and Systems for Video Technology, vol.16(9), pp.1142-1147, Sept. 2006. (download)
C. Wang, P. Xue, W. Lin, “Improved Super-resolution Reconstruction from Video”, IEEE Trans. Circuits and Systems for Video Technology, Vol.16(11), pp.1411-1422, Nov. 2006, (download)
E. Ong, W. Lin, B. Tye, M. Etoh, Fast Automatic Video Object Segmentation for Content-Based Applications, in Advances in Image and Video Segmentation, Chapter VII, ed. Y.J. Zhang, Idea Group, Inc., 2006. (download)
H. Gao, W. Lin, P. Xue, W.C. Siu, “Marker-Based Image Segmentation Relying on Disjoint Set Union”, Signal Processing: Image Communication, Vol. 21(2), pp. 100-112, 2006, (download)
P. Li, W. Lin, S. Rahardja, X. Lin, X.K. Yang, Z.G. Li, “Geometrically Determining the Leaky Bucket Parameters for Video Streaming over Constant Bit-Rate Channels”, Signal Processing: Image Communication, Vol. 20(2), pp.193-204, February 2005, (download)

Edited book

W. Lin, D. Tao, J. Kacprzyk, Z. Li, E. Izquierdo, H. Wang (eds.), Multimedia Analysis, Processing and Communications, Springer, 2011.
W. Lin, D. Xu, A. Ho, J. Wu, Y. He, J. Cai, M. Kankanhalli, M-T Sun (eds.), Advances in Multimedia Information Processing – PCM 2012, Springer, 2012.

Editorials

D. Xu, W. Lin, A. T. S. Ho, “Advances in multimedia content analysis and signal processing”, Journal of Signal Processing Systems, 74(1), 1-3, 2014.

Computationally Modelling for Environmental Monitoring

H. Y. Pak, H. T. Kieu, W. Lin, E. Khoo, A. W-K Law, “CoastalWQL: An open-source tool for drone-based mapping of coastal water quality using push broom hyperspectral imagery”, Remote Sensing, 16(4): 708, 2024. (download)
H. Y. Pak, A. W-K Law, W. Lin, “Retrieval of total suspended solids concentration from hyperspectral sensing using hierarchical Bayesian model aggregation for optimal multiple band ratio analysis”, Journal of Hydro-environment Research, 46: 1-18, Jan. 2023.
K. Gu, H. Liu, Z. Xia, J. Qiao, W. Lin, D. Thalmann, "PM2.5 Monitoring: Use Information Abundance Measurement and Wide & Deep Learning", IEEE Transactions on Neural Networks and Learning Systems, 32(10): 4278-4290, 2021 (download )
K. Gu, Z. Xia, J. Qiao, W. Lin, “Deep Dual-Channel Neural Network for Image-Based Smoke Detection”, IEEE Transactions on Multimedia, 22(2): 311 - 323, 2020. (download)
Use Deep Dual-Channel Neural Network (DCNN) in images for safety warning systems & fire prevention: to extract & then integrate low-level local texture features and high-level global contour info
K. Gu, J. Qiao, W. Lin, "Recurrent Air Quality Predictor Based on Meteorology- and Pollution-Related Factors", IEEE Transactions on Industrial Informatics, 14(9): 3946 – 3955, 2018. (download)
We hope to predict air quality as early as possible to enable meaningful corrective & preventive actions, but existing longer-term prediction is inaccurate; here is a simple yet effective solution: To use 1-hour regression model (as much-simplified LSTM networks, with less training requirement) to predict n-hours air quality recurrently with help of augmented data.

Green Computing via Neural Network (NN) Compression

Reduction of NN size without jeopardizing performance leads to environmentally friendly computing, in terms of energy consumption, battery life, device size, material used, processing speed, and so on,

K. Xu, Z. Wang, X. Geng, M. Wu, X. Li, W. Lin, “Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks”, IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17447-17457, 2023. (download)
Overall NN branch pruning --> branch pruning at each layer

Visual Understanding, Machine Learning & Applications

W. Yan, Y. Zhang, C. Tang, W. Zhou, W. Lin, “Anchor-Sharing and Clusterwise Contrastive Network for Multiview Representation Learning”, IEEE Trans. on Neural Networks and Learning Systems, in press (download)
H. Zhang, M. Leong, L. Li, W. Lin, “PeVL: Pose-Enhanced Vision-Language Model for Fine-Grained Human Action Recognition”, CVPR 2024
C. Chen, S. Zhou, L. Liao, H. Wu, W. Sun, Q. Yan, W. Lin, “Iterative Token Evaluation and Refinement for Real-World Super-Resolution”, AAAI 2024 (download)
H. Zhang, M. C. Leong, L. Li, W. Lin, “PGVT: Pose-Guided Video Transformer for Fine-Grained Action Recognition”, IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 6645-6656, 2024 (download)
F. Li, Y. Wu, H. Bai, W. Lin, R. Cong, J. Hou, J. Xiao, Y. Zhao, "Learning Detail-Structure Alternative Optimization for Blind Super-Resolution", IEEE Transactions on Multimedia, accepted (download)
Blind SR, i.e., without prior knowledge/assumption on degradation types/processes...
W. Yan, Y. Zhang, C. Lv, C. Tang, G. Yue, L. Liao, W. Lin, "GCFAgg: Global and Cross-view Feature Aggregation for Multi-view Clustering", CVPR 2023 (download)
W. Zhou, H. Zhang, W. Yan, W. Lin, “MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation, IEEE Trans. on Circuits and Systems for Video Technology, in press, 2023 (download)
W. Zhou, H. Zhang, W. Yan, W. Lin, “MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation, IEEE Trans. on Circuits and Systems for Video Technology, in press, 2023 (download)
G. Zhang, W. Lin, A. K. Chandran, X. Jing, “Complementary Networks for Person Re-Identification", Information Sciences, 633: 70–84, 2023 (download)
Y. Feng, X. Meng, F. Zhou, W. Lin, Z. Su, “Real-world Non-homogeneous Haze Removal by Sliding Self-attention Wavelet Network”, ”, IEEE Trans. on Circuits and Systems for Video Technology, 2023, in press. (download)
W. Yan, M. Gu, J. Ren, G. Yue, Z. Liu, J. Xu, W. Lin, “Collaborative Structure and Feature Learning for Multi-View Clustering”, Information Fusion, 98:101832, 2023. (download)
Y. Wu, F. Li, H. Bai, W. Lin, R. Cong, Y. Zhao, “Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution”, IEEE Trans. on Multimedia, in press. (download)
G. Zhang, Z. Luo, W. Lin, J. Xuan, “Inter-Intra Camera Identity Learning for Person Re-Identification with Training in Single Camera”, IEEE ICME 2023 (download)
G. Zhang, H. Zhang, W. Lin, A. K. Chandran, X. Jing, “Camera Contrast Learning for Unsupervised Person Re-Identification”, IEEE Trans. on Circuits and Systems for Video Technology, 2022, in press. (download)
F. Li, Y. Wu, H. Bai, W. Lin, R. Cong, J. Hou, J. Xiao, Y. Zhao, "Learning Detail-Structure Alternative Optimization for Blind Super-Resolution", IEEE Transactions on Multimedia, in press. (download)
X. Zhang, N. Cai, H. Zhang, Y. Zhang, J. Di, W. Lin, “AFD-former: A Hybrid Transformer with Asymmetric Flow Division for Synthesized View Quality Enhancement”, IEEE Trans. on Circuits and Systems for Video Technology, 2022, in press. (download)
Y. Wang, Y. Wang, Z. Chen, Y. Li, Z.Tang, W. Chu, J. Chen, W. Lin, K-K Ma, “CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes”, AAAI Conference on Artificial Intelligence (AAAI-22), accepted (download)
G. Zhang, Z. Luo, Y. Chen, Y. Zheng, W. Lin, “Illumination Unification for Person Re-Identification”, IEEE Trans. on Circuits and Systems for Video Tech., 2022 (download)
A data-augmentation solution when training data is with only one illumination condition... also with GAN style transfer in augmented images for training (because input images in practice will go through a GAN)
F. Li, A. Li, J. Qin, H. Bai, W. Lin, R. Cong, Y. Zhao，"SRInpaintor: When Super-Resolution Meets Transformer for Image Inpainting"，IEEE Transactions on Computational Imaging, 8: 743 - 758, 2022. (download)
K. H. Ooi, M. M. Liu, J. W. D. Tay, S. Y. Teo, P. Kaewsapsak, S. Jin, C. Lee, J. Hou, S. Mauer-Stroh, W. Lin, B. Yan, G. Yan, Y-G Gao, M. H. Tan， "An engineered CRISPR-Cas12a variant and DNA-RNA hybrid guides enable robust and rapid COVID-19 testing"， Nature Communications, 19 March 2021
G. Zhang, Y. Chen, W. Lin, A. K. Chandran, X. Jing, "Low Resolution Information Also Matters: Learning Multi-Resolution Representations for Person Re-Identification", 30th International Joint Conference on Artificial Intelligence (IJCAI), 2021 (download)
When a query image is with low resoltion (LR), the usual practice is to reconstruct a high resolution (HR) version as input to the networks; we demonstrate to achieve better performance if an LR version is also used since it contains original info that may be distorted or lost in the HR construction.
S. Mao, W. Lin, L. Jiao, S. Gou, J. Chen, “End-to-End Ensemble Learning by Exploiting the Correlation between Individuals and Weights”, IEEE Transactions on Cybernetics, accepted, 2020. (download )
A. Nguyen, W. Kim, J. Kim, W. Lin, S. Lee, “Video Frame Synthesis via Plug-and-Play Deep Locally Temporal Embedding”, IEEE Access, 2020. (download)
C. Deng, S. Wang, Z. Li, G. Huang, W. Lin, "Content Insensitive Blind Image Blurriness Assessment Using Weibull Statistics and Sparse Extreme Learning Machine," IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(3): 516–527, 2019. (download)
Z. Liu, G. Lin, S. Yang, F. Liu, W. Lin, W. L. Goh, “Conditional Spatial Expansion for Robust Curve Text Detection”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. (download)
Z. Liu, G. Lin, S. Yang, J. Feng, W. Lin, W. Goh, “Learning Markov Clustering Networks for Scene Text Detection”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 6936-6944, 2018 (download )
S. I. Niwas, W. Lin, C. K. Kwoh, C.-C. Jay Kuo, C. C. Sng, M. C. Aquino, Paul T. K. Chew, “Cross-examination for Angle-Closure Glaucoma Feature Detection”, IEEE J. Biomedical and Health Informatics, 20(1): 343-354, 2016. (download)
S. I. Niwas, W. Lin, X. Bai, C. K. Kwoh, C. C. Sng, M. C. Aquino, P.T.K. Chew, “Reliable Feature Selection Technique for Automated Angle Closure Glaucoma Mechanism Detection”, Journal of Medical Systems, Vol. 39, 2015. (download)
H. K. Y. Choi, W. Lin, S. C. Loon, C. Tan, W. Wong, J. See, Z. Gu, C. K. Kwoh, P. Chew, “Facial Scanning With a Digital Camera: A Novel Way of Screening for Primary Angle Closure”, Journal of Glaucoma, 17 Oct 2013 (download)
C. Zhang, W. Bian, D. Tao, W. Lin, “Discretized-Vapnik-Chervonenkis Dimension for Analyzing Complexity of Real Function Classes”, IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, Vol. 23(9), pp. 1461 – 1472, 2012 (download)
A. Wirawan, C. K. Kwoh, P. T. K. Chew, M. C. D. Aquino, S. C. Loon, J. See, C. Zheng, W. Lin, “Feature Selection for Computer-Aided Angle Closure Glaucoma Mechanism Detection”, Journal of Medical Imaging and Health Informatics, Vol. 2, 2012 (download)

Patents

W. Lin, B. Tye, E.P. Ong, “Dynamic Load-balancing between Two Processing Means for Real-time Video Encoding”, US patent 6,748,019 B1 (2004); Singapore patent 9902417-6 (1999).
Z. Lu, W. Lin, S. Yao, E. Ong, “Method for generating a quality oriented significance map for assessing the quality of an image or video”, US Patent number 7,590,287 B2 (2009), PCT Patent number WO2004/043054 A2 (2004).
Z. Lu, W. Lin, Z. Li, K. P. Lim, X. Lin, S. Rahardja, E. Ong, S. Yao, “Method for encoding a picture, computer program product and encoder”, US Patent Application No. 11/910,929, 2007.
E. Ong, W. Lin, Z. Lu, S. Yao, X. Yang, “Image and Video Quality Measurements”, Singapore Patent Application No. 200307620-5 (2003); PCT Patent number WO 2005/060272 A1 (2005).
E Ong, X Yang, W Lin, Z LU, S Yao, “METHOD AND SYSTEM FOR VIDEO QUALITY MEASUREMENTS”, European Patent 1,692,876, 2006; also WO2005055618 (A1) and US2007257988 (A1).
Z. Gu, W. Lin, B.-S. Lee, C. T. Lau, “Mode Dependent Templates for H.264/AVC Lossless Intra Coding”, US Provisional Patent Application, 61/553,372, October 2011.
A. Liu, W. Lin, H. Chen, P. Zhang, “Image Retargeting Quality Assessment”, filed for USA patent application no: 13/713,110, 2013.
Z. Chen, K. Fan, W. Lin, L. Duan, A. C. Kot, “Lossy Compression Framework for Intermediate Deep Feature Compression”, March 2019, filing for Singapore， USA and China patents.
G. Zhang, W. Lin, A. K. Chandran, X. Jing, “Cross-View Coupled Projections-Based Dictionary Learning For Person Re-Identification”, 2019, Singapore provisional patent application number 10201913787T.
D. Liu, W. Lin, A. K. Chandran, X. Jing, G. Lin, “Rule-Based Corrections For Multi-Object Tracking In Video”, Singapore provisional patent, 2020.
B. Zhao, W. Lin, “Image-Based Point Cloud Attribute Compression Using Two-Stage Dimensionality Transformation”, 2020, Singapore provisional patent application number 10202008512Q; PCT patent application.
C. Lv, W. Lin, “Approximate Intrinsic Voxel Structure For Point Cloud Resampling, Simplification And Mesh Reconstruction”, Singapore provisional patent application number 10202011637Y
G. Zhang, W. Lin, A. K. Chandran, X. Jing, “Low Resolution Information Also Matters: Learning Multi-Resolution Representations For Person Re-Identification”, 2021, Singapore provisional patent application number 10202104215R
G. Zhang, W. Lin, A. K. Chandran, X. Jing, “Complemental Human Parsing And Attention Guided Network For Person Re-Identification”, 2021, Singapore provisional patent application number 10202104214W.
J. Jin, L. Meng, S. Chen, W. Lin, Z. Chen, T-S Chang, Z-G Li, J-L Lin, “A New Image Compression Paradigm For Human And Machine Vision: High-Level Semantic And Low-Level Signal Combination”, Chinese patent application number 202110860294.5, 2021.
H. Wu, C. Chen, W. Sun, Q. Yan, W. Lin, “Video Quality Detection Method And Apparatus, Electronic Device And Storage Medium”, Singapore patent application number 10202200947V, Chinese Application No: 202210431764.0, 2022
C. Lv, W. Lin, J. Zheng, X. Jing, “Three Dimensional Data Processing Methods And Systems”, Singapore Application No: 10202301925X, 5 July 2023 (claiming priority to earlier SG application no. 10202250377N, “Curvature Smoothed Field-based Adaptively Isotropic Remeshing”, Singapore patent application 10202250377N， 2022).
H. Wu, C. Chen, J. Hou, L. Liao, A. Wang, W. Sun, Q. Yan, W. Lin, “Model Training And Image Processing Methods And Apparatuses, Electronic Device, And Storage Medium”, Singapore patent application, 10202250234Y， 2022
W. Wang, L. Xu, W. Lin, “Method and Apparatus for Quantization of Deep-learnt Visual Features Based on Bit Allocation in Feature Channels”, China Patent Application number 202210656786.7, 2022

Selected Publications

Perceptual Image Quality Assessment (IQA)

AI-Generated Image Quality Assessment

Image Aesthetic Considerations & Assessment

Evaluation of Other Signals

Just-Noticeable Difference (JND) Formulation

Visual Attention (VA) & Saliency Modeling

Visual Signal Representation for Machine Intelligence (MI)

Perceptual Visual Coding & Processing

3D Visual Model Acquisition & Processing

Other Visual Signal Coding & Processing

Computationally Modelling for Environmental Monitoring

Green Computing via Neural Network (NN) Compression

Visual Understanding, Machine Learning & Applications

Patents

Selected Publications

Perceptual Image Quality Assessment (IQA)

AI-Generated Image Quality Assessment

Image Aesthetic Considerations & Assessment

Evaluation of Other Signals

Just-Noticeable Difference (JND) Formulation

Visual Attention (VA) & Saliency Modeling

Visual Signal Represen​tation for Machine Intelligence (MI)

Perceptual Visual Coding & Processing

3D Visual Model Acquisition & Processing

Other Visual Signal Coding & Processing

Computationally Modelling for Environmental Monitoring

Green Computing via Neural Network (NN) Compression

Visual Understanding, Machine Learning & Applications

Patents

Visual Signal Representation for Machine Intelligence (MI)