I found this can't be done.

My work around I inserted a table beside the chart. With the same group names and color coded the same. A little header a top the table.