How to merge legends for color and shape when geom_hline has a separate (additional) entry in the color legend?

Question

I have the following code, which produces the following plot:

cols <- brewer.pal(n = 3, name = 'Dark2')

p4 <- ggplot(all.m, aes(x=xval, y=yval, colour = Approach, ymax = 0.95)) + theme_bw() + 
  geom_errorbar(aes(ymin= yval - se, ymax = yval + se), width=5, position=pd) + 
  geom_line(position=pd) + 
  geom_point(aes(shape=Approach, colour = Approach), size = 4) + 
  geom_hline(aes(yintercept = cp.best$slope, colour = "C2P"), show_guide = FALSE) + 
  scale_color_manual(name="Approach", breaks=c("C2P", "P2P", "CP2P"), values =  cols[c(1,3,2)]) + 
  scale_y_continuous(breaks = seq(0.4, 0.95, 0.05), "Test AUROC") +
  scale_x_continuous(breaks = seq(10, 150, by = 20), "# Number of Patient Samples in Training")
p4 <- p4 + theme(legend.direction = 'horizontal', 
      legend.position = 'top', 
      plot.margin = unit(c(5.1, 7, 4.5, 3.5)/2, "lines"), 
      text = element_text(size=15), axis.title.x=element_text(vjust=-1.5), axis.title.y=element_text(vjust=2))   
p4 <- p4 + guides(colour=guide_legend(override.aes=list(shape=c(NA,17,16))))

p4

When I try show_guide = FALSE in geom_point, the shape of the point in the upper legend are all set to default solid circles.

How can I make the lower legend to disappear, without affecting the upper legend?

Answer 1

This is a solution, complete with reproducible data:

library("ggplot2")
library("grid")
library("RColorBrewer")

cp2p <- data.frame(xval = 10 * 2:15, yval = cumsum(c(0.55, rnorm(13, 0.01, 0.005))), Approach = "CP2P", stringsAsFactors = FALSE)
p2p <- data.frame(xval = 10 * 1:15, yval = cumsum(c(0.7, rnorm(14, 0.01, 0.005))), Approach = "P2P", stringsAsFactors = FALSE)

pd <- position_dodge(0.1)
cp.best <- list(slope = 0.65)

all.m <- rbind(p2p, cp2p)
all.m$Approach <- factor(all.m$Approach, levels = c("C2P", "P2P", "CP2P"))
all.m$se <- rnorm(29, 0.1, 0.02)
all.m[nrow(all.m) + 1, ] <- all.m[nrow(all.m) + 1, ] # Creates a new row filled with NAs
all.m$Approach[nrow(all.m)] <- "C2P"
cols <- brewer.pal(n = 3, name = 'Dark2')

p4 <- ggplot(all.m, aes(x=xval, y=yval, colour = Approach, ymax = 0.95)) + theme_bw() + 
  geom_errorbar(aes(ymin= yval - se, ymax = yval + se), width=5, position=pd) + 
  geom_line(position=pd) + 
  geom_point(aes(shape=Approach, colour = Approach), size = 4, na.rm = TRUE) + 
  geom_hline(aes(yintercept = cp.best$slope, colour = "C2P")) + 
  scale_color_manual(values = c(C2P = cols[1], P2P = cols[2], CP2P = cols[3])) + 
  scale_shape_manual(values = c(C2P = NA, P2P = 16, CP2P = 17)) +
  scale_y_continuous(breaks = seq(0.4, 0.95, 0.05), "Test AUROC") +
  scale_x_continuous(breaks = seq(10, 150, by = 20), "# Number of Patient Samples in Training")
p4 <- p4 + theme(legend.direction = 'horizontal', 
                 legend.position = 'top', 
                 plot.margin = unit(c(5.1, 7, 4.5, 3.5)/2, "lines"), 
                 text = element_text(size=15), axis.title.x=element_text(vjust=-1.5), axis.title.y=element_text(vjust=2))   
p4

The trick is to make sure that all of the desired levels of all.m$Approach appear in all.m, even if one of them gets dropped out of the graph. The warning about the omitted point is suppressed by the na.rm = TRUE argument to geom_point.

Answer 2

Short answer:
Just add a dummy geom_point layer (transparent points) where shape is mapped to the same level as in geom_hline.

geom_point(aes(shape = "int"), alpha = 0) 

Longer answer:
Whenever possible, ggplot merges / combines legends of different aesthetics. For example, if colour and shape is mapped to the same variable, then the two legends are combined into one.

I illustrate this using simple data set with 'x', 'y' and a grouping variable 'grp' with two levels:

df <- data.frame(x = rep(1:2, 2), y = 1:4, grp = rep(c("a", "b"), each = 2))

First we map both color and shape to 'grp'

ggplot(data = df, aes(x = x, y = y, color = grp, shape = grp)) +
  geom_line() +
  geom_point(size = 4)

Fine, the legends for the aesthetics, color and shape, are merged into one.

Then we add a geom_hline. We want it to have a separate color from the geom_lines and to appear in the legend. Thus, we map color to a variable, i.e. put color inside aes of geom_hline. In this case we do not map the color to a variable in the data set, but to a constant. We may give the constant a desired name, so we don't need to rename the legend entries afterwards.

ggplot(data = df, aes(x = x, y = y, color = grp, shape = grp)) +
  geom_line() +
  geom_point(size = 4) +
  geom_hline(aes(yintercept = 2.5, color = "int"))

Now two legends appears, one for the color aesthetics of geom_line and geom_hline, and one for the shape of the geom_points. The reason for this is that the "variable" which color is mapped to now contains three levels: the two levels of 'grp' in the original data, plus the level 'int' which was introduced in the geom_hline aes. Thus, the levels in the color scale differs from those in the shape scale, and by default ggplot can't merge the two scales into one legend.

How to combine the two legends?

One possibility is to introduce the same, additional level for shape as for color by using a dummy geom_point layer with transparent points (alpha = 0) so that the two aesthetics contains the same levels:

ggplot(data = df, aes(x = x, y = y, color = grp, shape = grp)) +
  geom_line() +
  geom_point(size = 4) +
  geom_hline(aes(yintercept = 2.5, color = "int")) +
  geom_point(aes(shape = "int"), alpha = 0) # <~~~~ a blank geom_point

Another possibility is to convert the original grouping variable to a factor, and add the "geom_hline level" to the original levels. Then use drop = FALSE in scale_shape_discrete to include "unused factor levels from the scale":

datadf$grp <- factor(df$grp, levels = c(unique(df$grp), "int"))

ggplot(data = df, aes(x = x, y = y, color = grp, shape = grp)) +
  geom_line() +
  geom_point(size = 4) +
  geom_hline(aes(yintercept = 2.5, color = "int")) +
  scale_shape_discrete(drop = FALSE)

Then, as you already know, you may use the guides function to "override" the shape aesthetics in the legend, and remove the shape from the geom_hline entry by setting it to NA:

guides(colour = guide_legend(override.aes = list(shape = c(16, 17, NA))))