Alternatively, you could say

Map<Person, List<Relation>> relationships;

At this point, we're talking about modeling the person and their
relationships as a Graph, and it might make sense to consider other
data model approaches.

Perhaps instead of the above map, we have:
final class Relation {
   /* enum RelationType is defined somewhere else.
      It defines the reciprocal as well:
      RelationType.PARENT.reciprocal() == RelationType.CHILD;
      Note: we leave gender out of the relationship type for many
reasons.
    */
  private RelationType type;
  private final Person person;
  private Person relative;

  public Relation(RelationType type, Person relative, Person person)
{
    /* initialize here */
  }

  /* snip other constructors/accessors */

  Relation reciprocal() {
     return new Relation(type.reciprocal(), person, relative);
  }
}

class Person {
  final Collection<Relation> relationships =
             new ArrayList<Relation>();
}

What we've ended up with here is a specific class of a Node (person)
and Edge (Relation).

So, my point is that there are many ways to model this.  It all
depends on what you need from your model.  A good engineer will likely
start with the easiest/simplest approach (such as the OP solution),
and refactor into the other models as they become appropriate for the
project.