SteLLA: A Structured Grading System Using LLMs with RAG

Hefei Qiu; Brian White; Ashley Ding; Reinaldo Costa; Ali Hachem; Wei Ding; Ping Chen

doi:10.1109/BigData62323.2024.10825385

SteLLA: A Structured Grading System Using LLMs with RAG

Hefei Qiu
, Brian White
, Ashley Ding
, Reinaldo Costa
, Ali Hachem
, Wei Ding
, Ping Chen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Original language	English
Title of host publication	Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024
Editors	Wei Ding, Chang-Tien Lu, Fusheng Wang, Liping Di, Kesheng Wu, Jun Huan, Raghu Nambiar, Jundong Li, Filip Ilievski, Ricardo Baeza-Yates, Xiaohua Hu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	8154-8163
Number of pages	10
ISBN (Electronic)	9798350362480
DOIs	https://doi.org/10.1109/BigData62323.2024.10825385
State	Published - 2024
Event	2024 IEEE International Conference on Big Data, BigData 2024 - Washington, United States Duration: Dec 15 2024 → Dec 18 2024

Publication series

Name	Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024

Conference

Conference	2024 IEEE International Conference on Big Data, BigData 2024
Country/Territory	United States
City	Washington
Period	12/15/24 → 12/18/24

ASJC Scopus Subject Areas

Artificial Intelligence
Computer Networks and Communications
Computer Science Applications
Information Systems
Information Systems and Management
Safety, Risk, Reliability and Quality
Modeling and Simulation

Keywords

LLM-based ASAG system
QA-based evaluation
RAG
structured evaluation

Access to Document

10.1109/BigData62323.2024.10825385

Cite this

Qiu, H., White, B., Ding, A., Costa, R., Hachem, A., Ding, W., & Chen, P. (2024). SteLLA: A Structured Grading System Using LLMs with RAG. In W. Ding, C.-T. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, & X. Hu (Eds.), Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024 (pp. 8154-8163). (Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData62323.2024.10825385

SteLLA: A Structured Grading System Using LLMs with RAG. / Qiu, Hefei; White, Brian; Ding, Ashley et al.
Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024. ed. / Wei Ding; Chang-Tien Lu; Fusheng Wang; Liping Di; Kesheng Wu; Jun Huan; Raghu Nambiar; Jundong Li; Filip Ilievski; Ricardo Baeza-Yates; Xiaohua Hu. Institute of Electrical and Electronics Engineers Inc., 2024. p. 8154-8163 (Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Qiu, H, White, B, Ding, A, Costa, R, Hachem, A, Ding, W & Chen, P 2024, SteLLA: A Structured Grading System Using LLMs with RAG. in W Ding, C-T Lu, F Wang, L Di, K Wu, J Huan, R Nambiar, J Li, F Ilievski, R Baeza-Yates & X Hu (eds), Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024. Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024, Institute of Electrical and Electronics Engineers Inc., pp. 8154-8163, 2024 IEEE International Conference on Big Data, BigData 2024, Washington, United States, 12/15/24. https://doi.org/10.1109/BigData62323.2024.10825385

Qiu H, White B, Ding A, Costa R, Hachem A, Ding W et al. SteLLA: A Structured Grading System Using LLMs with RAG. In Ding W, Lu CT, Wang F, Di L, Wu K, Huan J, Nambiar R, Li J, Ilievski F, Baeza-Yates R, Hu X, editors, Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024. Institute of Electrical and Electronics Engineers Inc. 2024. p. 8154-8163. (Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024). doi: 10.1109/BigData62323.2024.10825385

Qiu, Hefei ; White, Brian ; Ding, Ashley et al. / SteLLA : A Structured Grading System Using LLMs with RAG. Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024. editor / Wei Ding ; Chang-Tien Lu ; Fusheng Wang ; Liping Di ; Kesheng Wu ; Jun Huan ; Raghu Nambiar ; Jundong Li ; Filip Ilievski ; Ricardo Baeza-Yates ; Xiaohua Hu. Institute of Electrical and Electronics Engineers Inc., 2024. pp. 8154-8163 (Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024).

@inproceedings{5b94f0af8972431f97e581a4dc7b616c,

title = "SteLLA: A Structured Grading System Using LLMs with RAG",

keywords = "LLM-based ASAG system, QA-based evaluation, RAG, structured evaluation",

author = "Hefei Qiu and Brian White and Ashley Ding and Reinaldo Costa and Ali Hachem and Wei Ding and Ping Chen",

note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 2024 IEEE International Conference on Big Data, BigData 2024 ; Conference date: 15-12-2024 Through 18-12-2024",

year = "2024",

doi = "10.1109/BigData62323.2024.10825385",

language = "English",

series = "Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "8154--8163",

editor = "Wei Ding and Chang-Tien Lu and Fusheng Wang and Liping Di and Kesheng Wu and Jun Huan and Raghu Nambiar and Jundong Li and Filip Ilievski and Ricardo Baeza-Yates and Xiaohua Hu",

booktitle = "Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024",

}

TY - GEN

T1 - SteLLA

T2 - 2024 IEEE International Conference on Big Data, BigData 2024

AU - Qiu, Hefei

AU - White, Brian

AU - Ding, Ashley

AU - Costa, Reinaldo

AU - Hachem, Ali

AU - Ding, Wei

AU - Chen, Ping

PY - 2024

Y1 - 2024

KW - LLM-based ASAG system

KW - QA-based evaluation

KW - RAG

KW - structured evaluation

UR - https://www.scopus.com/pages/publications/85217984039

UR - https://www.scopus.com/pages/publications/85217984039#tab=citedBy

U2 - 10.1109/BigData62323.2024.10825385

DO - 10.1109/BigData62323.2024.10825385

M3 - Conference contribution

AN - SCOPUS:85217984039

T3 - Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024

SP - 8154

EP - 8163

BT - Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024

A2 - Ding, Wei

A2 - Lu, Chang-Tien

A2 - Wang, Fusheng

A2 - Di, Liping

A2 - Wu, Kesheng

A2 - Huan, Jun

A2 - Nambiar, Raghu

A2 - Li, Jundong

A2 - Ilievski, Filip

A2 - Baeza-Yates, Ricardo

A2 - Hu, Xiaohua

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 15 December 2024 through 18 December 2024

ER -

SteLLA: A Structured Grading System Using LLMs with RAG

Publication series

Conference

ASJC Scopus Subject Areas

Keywords

Access to Document

Other files and links

Fingerprint

Cite this