evaluate_links() measures how well each index of a predict_links()
result recovers a set of known edges. For each index it returns the area
under the ROC curve (AUC), the average precision, and the precision among
the top k predictions.
The AUC is the probability that a known edge scores higher than a pair
that is not one (ties count half), so 0.5 is chance level. Average
precision is the mean of the precisions at the rank of each known edge,
and precision\(@\)k the share of known edges among the k
highest-scoring pairs. Only the pairs listed in the predictions are
evaluated: with exclude_existing = TRUE, the default of
predict_links(), the network's current edges are not scored, so the
known edges should be ones held out of the network (a later period, a
removed subset), and top_n limits the evaluation to the pairs it kept.
The AUC is NA when the evaluated pairs include no known edge, or
nothing but known edges.
Usage
evaluate_links(pred, true_edges, k = c(5L, 10L, 20L))Arguments
- pred
A
predict_links()result.- true_edges
The known edges: a data frame with columns
fromandto(every row is an edge, whatever its other columns, such as a weight, hold), or a matrix whose non-zero cells are edges. Its row names name both its rows and its columns (column names are not read), so its columns must be in the same order as its rows (column names that differ from the row names raise a warning of classnestimate_ignored_argument); without row names, the nodes ofpredare used. For a directed prediction, pairs are matched byfromandto, direction included; for an undirected one, in either order, so(C, A)matches the listed pair(A, C).- k
The cut-offs for precision\(@\)k. Default
c(5, 10, 20). Aklarger than an index's number of predictions givesNA.
Value
A data frame with one row per index: method, auc,
average_precision (0 when no evaluated pair is a known edge), and
one precision_at_<k> column per value of k. A prediction with no
listed pairs (every possible link already exists) gives NULL.
Examples
seqs <- data.frame(
V1 = c("A", "B", "C", "D", "A", "C", "E", "B"),
V2 = c("B", "C", "D", "E", "C", "E", "A", "D"),
V3 = c("C", "D", "E", "A", "D", "A", "B", "E")
)
net <- build_network(seqs, method = "relative")
pred <- predict_links(net, exclude_existing = FALSE)
# Evaluate against the network's own edges as the known truth
evaluate_links(pred, extract_edges(net, threshold = 0.001))
#> method auc average_precision precision_at_5
#> 1 common_neighbors 0.5833333 0.4618602 0.4
#> 2 resource_allocation 0.5833333 0.4618602 0.4
#> 3 adamic_adar 0.5833333 0.4618602 0.4
#> 4 jaccard 0.3333333 0.3242384 0.0
#> 5 preferential_attachment 0.6875000 0.7002999 0.8
#> 6 katz 0.7916667 0.6196293 0.4
#> precision_at_10 precision_at_20
#> 1 0.4 0.4
#> 2 0.4 0.4
#> 3 0.4 0.4
#> 4 0.3 0.4
#> 5 0.5 0.4
#> 6 0.7 0.4