The task of grouping strokes into different categories is an essential processing step in the automatic analysis of online
handwritten documents. The technical challenge originates from the variation of the handwriting style, content heterogeneity
and lack of prior layout knowledge. In this work, we propose the edge graph attention network (EGAT) to address the stroke
classification problem. In this framework, the stroke classification problem is formulated as a node classification problem in
a relational graph, which is constructed based on the temporal and spatial relationship of strokes. Then distributed node and
edge features for classification are learned by stacking of multiple edge graph attention layers, in which various attention
mechanisms are exploited to aggregate information between neighborhood nodes. In the task of text/nontext classification,
the proposed model achieves accuracies 98.65% and 98.90% on the IAMOnDo and Kondate datasets, respectively. In the
task of multi-class classification, the achieved accuracies are 95.81%, 97.36% and 99.05% on the IAMOnDo, FC and FA
datasets, respectively. In addition, we conduct ablation experiments to quantitatively and qualitatively evaluate the key
modules of our model.