We use the highly acclaimed ai-scientist for full paper evaluation.
First, download ai-scientist
We validated ai-scientist reliablity as a full paper evaluator by analyzing how closely its predicted decision (accept/reject) of a given conference paper aligns with the actual. The code is in validate_ai_scientist.py
We evaluated XtraGPT modified papers with ai_sicentist.py. The results are stored in paper_results
We analyze improvements in acceptance rate in analyze_improvement.py and plots used in the diagram detailing improvements to different metrics such as soundness, presentation, contribution are found in xtragpt_plots. The code for the plots can be found in plot.py