There are basically two dimensions upon which judges are evaluated.
One is simple competence. Does a judge have a suitably judicial demeanor, act ethically, get his or her work done in a timely manner, and understand the law?
The other is ideological. Judging provides far room for discretionary decision making than the "judge as umpire" metaphor would suggest. Some judges are more conservative, some are more liberal, and judges tend to moderate their ruling when they are on panels of multiple judges with different ideologiies. A new study compares different ways of evaluating judicial ideology empirically.