High temperature refractory alloys of tungsten, molybdenum and tantalum are under evaluation for use as structural materials in the Advanced Power Extraction Program (APEX) because of their ability to accommodate high wall loading and high temperature coolant. However, such materials tend to have undesirable safety and environmental characteristics relative to conventional reduced-activation fusion materials. These alloys have high decay heat and in some cases their activation results in the production of long-lived isotopes that would disqualify the material from being disposed of as low level waste. In addition, some of the alloys have oxides that are very volatile, which could be mobilized in accident scenarios in which air ingress is a concern. In this paper we compare the safety and environmental characteristics of these alloys with their low activation cousin (vanadium) in terms of decay heat, oxidation driven mobilization in air, and waste management.