Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models

Wenxing Liu; Ming Cheng; Yueran Pan; Lynn Yuan; Suxiu Hu; Ming Li; Songtian Zeng

doi:10.1007/978-981-99-8558-6_22

Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models

Wenxing Liu
, Ming Cheng
, Yueran Pan
, Lynn Yuan
, Suxiu Hu
, Ming Li
, Songtian Zeng

Curriculum & Instruction

Wuhan University
Co.
Duke Kunshan University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that has gained global attention due to its prevalence. Clinical assessment measures rely heavily on manual scoring conducted by specialized physicians. However, this approach exhibits subjectivity and challenges in regions without sufficient medical resources. This study presents paradigms designed to automatically evaluate various aspects of social skills in children with ASD, utilizing our multiview and multimodal behavior system. Moreover, we propose a new pipeline to predict autism-related social skill scores using the language-image pre-training model. Our multimodal behavioral database comprises 12 subjects (511 videos with labeled social skill scores). Finally, we achieve 81.46% accuracy on the paradigm success prediction task and 69.23% accuracy on the social skill ability scoring task. The results demonstrate that the language-image pre-training model can effectively introduce domain knowledge into video assessment tasks.

Original language	English
Title of host publication	Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings
Editors	Qingshan Liu, Hanzi Wang, Rongrong Ji, Zhanyu Ma, Weishi Zheng, Hongbin Zha, Xilin Chen, Liang Wang
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	260-271
Number of pages	12
ISBN (Print)	9789819985579
DOIs	https://doi.org/10.1007/978-981-99-8558-6_22
State	Published - 2024
Event	6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023 - Xiamen, China Duration: Oct 13 2023 → Oct 15 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14437 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023
Country/Territory	China
City	Xiamen
Period	10/13/23 → 10/15/23

ASJC Scopus Subject Areas

Theoretical Computer Science
General Computer Science

Keywords

Autism Spectrum Disorder
Behavior
Language-Image Pre-training Models
Social Skills Assessment

Access to Document

10.1007/978-981-99-8558-6_22

Cite this

Liu, W., Cheng, M., Pan, Y., Yuan, L., Hu, S., Li, M., & Zeng, S. (2024). Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models. In Q. Liu, H. Wang, R. Ji, Z. Ma, W. Zheng, H. Zha, X. Chen, & L. Wang (Eds.), Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings (pp. 260-271). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14437 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-8558-6_22

Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models. / Liu, Wenxing; Cheng, Ming; Pan, Yueran et al.
Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings. ed. / Qingshan Liu; Hanzi Wang; Rongrong Ji; Zhanyu Ma; Weishi Zheng; Hongbin Zha; Xilin Chen; Liang Wang. Springer Science and Business Media Deutschland GmbH, 2024. p. 260-271 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14437 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Liu, W, Cheng, M, Pan, Y, Yuan, L, Hu, S, Li, M & Zeng, S 2024, Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models. in Q Liu, H Wang, R Ji, Z Ma, W Zheng, H Zha, X Chen & L Wang (eds), Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14437 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 260-271, 6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023, Xiamen, China, 10/13/23. https://doi.org/10.1007/978-981-99-8558-6_22

Liu W, Cheng M, Pan Y, Yuan L, Hu S, Li M et al. Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models. In Liu Q, Wang H, Ji R, Ma Z, Zheng W, Zha H, Chen X, Wang L, editors, Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 260-271. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-99-8558-6_22

Liu, Wenxing ; Cheng, Ming ; Pan, Yueran et al. / Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models. Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings. editor / Qingshan Liu ; Hanzi Wang ; Rongrong Ji ; Zhanyu Ma ; Weishi Zheng ; Hongbin Zha ; Xilin Chen ; Liang Wang. Springer Science and Business Media Deutschland GmbH, 2024. pp. 260-271 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c4505ce896494613acd8d6936c7d7ef4,

title = "Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models",

abstract = "Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that has gained global attention due to its prevalence. Clinical assessment measures rely heavily on manual scoring conducted by specialized physicians. However, this approach exhibits subjectivity and challenges in regions without sufficient medical resources. This study presents paradigms designed to automatically evaluate various aspects of social skills in children with ASD, utilizing our multiview and multimodal behavior system. Moreover, we propose a new pipeline to predict autism-related social skill scores using the language-image pre-training model. Our multimodal behavioral database comprises 12 subjects (511 videos with labeled social skill scores). Finally, we achieve 81.46\% accuracy on the paradigm success prediction task and 69.23\% accuracy on the social skill ability scoring task. The results demonstrate that the language-image pre-training model can effectively introduce domain knowledge into video assessment tasks.",

keywords = "Autism Spectrum Disorder, Behavior, Language-Image Pre-training Models, Social Skills Assessment",

author = "Wenxing Liu and Ming Cheng and Yueran Pan and Lynn Yuan and Suxiu Hu and Ming Li and Songtian Zeng",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2024.; 6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023 ; Conference date: 13-10-2023 Through 15-10-2023",

year = "2024",

doi = "10.1007/978-981-99-8558-6\_22",

language = "English",

isbn = "9789819985579",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "260--271",

editor = "Qingshan Liu and Hanzi Wang and Rongrong Ji and Zhanyu Ma and Weishi Zheng and Hongbin Zha and Xilin Chen and Liang Wang",

booktitle = "Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings",

}

TY - GEN

T1 - Assessing the Social Skills of Children with Autism Spectrum Disorder via Language-Image Pre-training Models

AU - Liu, Wenxing

AU - Cheng, Ming

AU - Pan, Yueran

AU - Yuan, Lynn

AU - Hu, Suxiu

AU - Li, Ming

AU - Zeng, Songtian

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2024.

PY - 2024

Y1 - 2024

N2 - Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that has gained global attention due to its prevalence. Clinical assessment measures rely heavily on manual scoring conducted by specialized physicians. However, this approach exhibits subjectivity and challenges in regions without sufficient medical resources. This study presents paradigms designed to automatically evaluate various aspects of social skills in children with ASD, utilizing our multiview and multimodal behavior system. Moreover, we propose a new pipeline to predict autism-related social skill scores using the language-image pre-training model. Our multimodal behavioral database comprises 12 subjects (511 videos with labeled social skill scores). Finally, we achieve 81.46% accuracy on the paradigm success prediction task and 69.23% accuracy on the social skill ability scoring task. The results demonstrate that the language-image pre-training model can effectively introduce domain knowledge into video assessment tasks.

AB - Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that has gained global attention due to its prevalence. Clinical assessment measures rely heavily on manual scoring conducted by specialized physicians. However, this approach exhibits subjectivity and challenges in regions without sufficient medical resources. This study presents paradigms designed to automatically evaluate various aspects of social skills in children with ASD, utilizing our multiview and multimodal behavior system. Moreover, we propose a new pipeline to predict autism-related social skill scores using the language-image pre-training model. Our multimodal behavioral database comprises 12 subjects (511 videos with labeled social skill scores). Finally, we achieve 81.46% accuracy on the paradigm success prediction task and 69.23% accuracy on the social skill ability scoring task. The results demonstrate that the language-image pre-training model can effectively introduce domain knowledge into video assessment tasks.

KW - Autism Spectrum Disorder

KW - Behavior

KW - Language-Image Pre-training Models

KW - Social Skills Assessment

UR - https://www.scopus.com/pages/publications/85181765346

UR - https://www.scopus.com/pages/publications/85181765346#tab=citedBy

U2 - 10.1007/978-981-99-8558-6_22

DO - 10.1007/978-981-99-8558-6_22

M3 - Conference contribution

AN - SCOPUS:85181765346

SN - 9789819985579

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 260

EP - 271

BT - Pattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings

A2 - Liu, Qingshan

A2 - Wang, Hanzi

A2 - Ji, Rongrong

A2 - Ma, Zhanyu

A2 - Zheng, Weishi

A2 - Zha, Hongbin

A2 - Chen, Xilin

A2 - Wang, Liang

PB - Springer Science and Business Media Deutschland GmbH

T2 - 6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023

Y2 - 13 October 2023 through 15 October 2023

ER -