AFFECTIVE AI · MULTIMODAL INTELLIGENCE

Intelligence,
shaped by people.

I'm Junjie Xu. Ph.D. researcher at ECNU.

I connect sound, images and human experience through multimodal alignment, affective understanding and controllable generation.

Portrait of Junjie Xu
Junjie XuEast China Normal University
Based in Shanghai · Thinking beyond modalities
4 / 14First-author / published papers
2Funded research projects led
14Patent applications · co-inventor
NEXT CHAPTERECNU NTUCSC-funded visit · Nov 2026–Nov 2027

A CONNECTED RESEARCH PATH

Technical depth.
A human perspective.

My work brings three capabilities together — from understanding a signal to building something meaningful with it.

01

Connect the modalities.

Audio, language and images — with alignment that preserves the details that matter.

MARS · IPM
02

Understand the human signal.

Bring affect, context and human feedback into how models interpret and generate.

ACM TALLIP · IJCNN
03

Make research useful.

Lead projects from research questions to prototypes and human-centered applications.

2 funded projects led

SELECTED WORK / 01–03

A few works.
A clearer picture.

Explore published research
01 / CROSS-MODAL GENERATIONPublished · First author

MARSFrom sound to a visual world.

Fine-grained audio–image alignment brings semantic detail and aesthetic quality into the same generation process.

Information Processing & Management
02 / AFFECTIVE UNDERSTANDINGPublished

Understanding sentiment,
in its context.

First-author research in aspect-level sentiment analysis, with published work in ACM TALLIP and IJCNN.

ACM TALLIP · IJCNN
03 / RESEARCH INTO PRACTICEProject lead

Technology with
a little more care.

Led the ECNU multimodal digital-support project, rated excellent at completion. Exploring how affective generation can support everyday experience.

ECNU · 2024–2025 · Excellent

WHAT'S NEXT

Where this work
could go next.

Prospective directions grounded in my current work. Open to conversations and research collaboration.

01

HUMAN–AI COLLABORATION

Affective, trustworthy agents.

How can affective cues and human feedback support appropriate trust, oversight and long-term collaboration?

Affective computing → agent collaboration
02

CONTROLLABLE GENERATION

Generation with human intent.

Build on audio–image alignment toward multimodal generation shaped by emotional intent and human preferences.

Cross-modal alignment → preference learning
03

HUMAN-CENTERED APPLICATIONS

Digital support, thoughtfully built.

Turn research into carefully evaluated experiences for digital well-being and educational support.

Project leadership → real-world evaluation

A new research chapter at NTU.CSC-funded visiting research with Prof. Eric Cambria, scheduled for November 2026–November 2027.

PUBLISHED RESEARCH

Selected publications.

All 14 published papers
  1. FIRST AUTHOR

    MARS: Multimodal-Assisted Refined Semantic Alignment

    Junjie Xu, Xingjiao Wu, Zihao Zhang, Shuwen Yang, Tianlong Ma, Daoguo Dong, Liang He

    Information Processing & Management 63 (1), 104292Published
  2. FIRST AUTHOR

    Bidirectional Directed Acyclic Graph Neural Network for Aspect-level Sentiment Classification

    Junjie Xu, Luwei Xiao, Anran Wu, Tianlong Ma, Daoguo Dong, Liang He

    ACM Transactions on Asian and Low-Resource Language Information ProcessingPublished
INNOVATION14 patent applications

Co-inventor; public application numbers are listed for verification.

  1. 一种基于Kapur阈值法的黑色素瘤皮肤镜图像分割方法

    CN 118840378 A · 2024-10-25

  2. 一种基于人在回路与视觉基础模型的医学图像增强方法

    CN 120298246 A · 2025-07-11

  3. 一种基于人在回路中标注样本的医学图像分割方法

    CN 118485681 A · 2024-08-13

  4. 一种基于偏好传递性的人在回路强化学习数据增强方法

    CN 118916133 A · 2024-11-08

  5. 一种基于图像多属性建模的美学评估方法

    CN 117422962 A · 2024-01-19

  6. 一种基于多层次对抗学习的图像质量评估方法

    CN 120198411 A · 2025-06-24

  7. 一种基于多模态交叉注意力与动态特征融合的音乐生成方法

    CN 120316301 A · 2025-07-15

  8. 一种基于对抗神经网络的音频驱动图像编辑方法

    CN 118524266 A · 2024-08-20

  9. 一种基于情感驱动和人类专家反馈的机器行为学习方法

    CN 120318634 A · 2025-07-15

  10. 一种基于情感驱动的多乐器分离协同音乐生成方法

    CN 120319204 A · 2025-07-15

  11. 一种基于扩散模型的小样本字体生成方法

    CN 118898549 A · 2024-11-05

  12. 一种基于有向无环图网络的方面级情感分类方法

    CN 116127072 A · 2023-05-16

  13. 一种基于注意力图卷积网络的多模态融合机械缺陷检测方法

    CN 114170477 A · 2022-03-11

  14. 基于人在回路与视觉基础模型的少样本医学图像分割方法

    CN 120298698 A · 2025-07-11

RECOGNITIONHonors & project experience
  • 2026
    China International College Students' Innovation Competition

    Higher-Education Main Track · Graduate Creative Group · Project Lead — national round in progress

  • 2025
    Kuanrui Talent Scholarship

    ECNU School of Computer Science and Technology

  • 2023
    Shanghai Outstanding Graduate

    Shanghai Municipal Education Commission

  • 2023
    National Third Prize · Challenge Cup

    China Association for Science and Technology

  • 2022
    Huaxin Scholarship

    ECNU School of Computer Science and Technology

LET'S CONNECT

Good research starts
with a conversation.

For research, collaboration, or a shared question.

Get in touch jjxu_dr@stu.ecnu.edu.cn