Just a guess…
If, in setting up each row, you’re following links from that row to other rows, then you’re incurring new database reads, on on each link.
Links generally resolve faster on the Server than on the Client. (The Server’s closer to the data.) You might save time by doing all of the link-following, for the entire set of rows, on the Server, and returning only the finished result to the Client as a single list.
Alternatively, a different data layout or visual layout might save some time.