In general you can only use policy groups for specific Endpoint Servers, not set them up against Agents directly.
However, if you want to utilize policy groups you can assign a set of agents to multiple Endpoint Servers and then each Endpoint Server to a specific policy group. This will result in the Endpoint Agents assigned to that specific server work off the policy within the assigned policy group.