07 June 2007
Trip Report
Department of Defense
Human Factors Engineering Technical Advisory Group
(DOD HFE TAG) Meeting #57
14-17 May
2007
The 57th meeting of the DoD HFE TAG was held in Portsmouth,
VA. The meeting was chaired by Mr. Adrian Salinas, 311th Human
Systems Wing, Brooks AFB, TX. The theme of the meeting was Reducing Risk
Through Collaboration. Approximately 100 people attended the
meeting, representing the US Army, US Navy, US Air Force, DARPA, NASA, FAA, DoD
Laboratories, Canadian DRDC, several human factors-related technical societies
and industry associations. Several additional personnel representing industry
and academia attended as invited speakers.
Four items are attached:
· Background of the DoD HFE TAG, attachment (1)
· TAG-57 Theme, Attachment (2)
· Program Summary, attachment (3)
· DoD HFE TAG Operating Board, attachment (4)
· TAG attendees, attachment (5)
· DoD HFE TAG Policies, attachment (6)
Plenary Session Presentations
The DoD
HFE TAG Chair for the 57th meeting, Adrian
Salinas, welcomed attendees to the meeting
and elaborated briefly on the meeting theme. He stressed that the HFE community needs to understand,
communicate, participate and collaborate, as well as share tools, methods and
research products. We also need to
ensure that our successful products transition to use.
Reducing Risks by Developing Software Human Factors Metrics and Methods - As a Team was presented by Dr. Patricia Chalmers, Science Advisor, JFCOM (patricial.chalmers@jfcom.mil). Dr Chalmers is usually the only HFE person on the team. What she usually hears is things like: You can’t do that! What about the Cost and Schedule? How many subjects do you want? I don’t care what you do, just get ‘er done! What we all have in common are tight schedules, funding cuts and the Warfighter. We need to adept a business model, not just a research model. We need demonstrate return on investment. We also need metrics – measures to assess benefits to our stakeholders (senior managers, fund providers, project managers, software coders, testers and users. Some examples of practical types metrics we need to consider:
- Dollars saved in reduced calls to the help desk
- Hours saved in re-coding software
- Time to complete tasks
- Time and money saved due to errors avoided
- Time and money saved due to accidents avoided
- Latency in minutes
- Number of positive comments on new systems
When defining experiments, remember that our war-fighters depend upon the validity of pour experiments. Everyone needs to cope with funding cuts, down-sizing, and egos. But, if you aren’t developing capabilities for as many services and agencies as possible, you may be developing tomorrow’s problems.
Navy Human Systems Integration Update was presented by Mr. Richard M. Etheridge, OP-125 (richard.etheridge@navy.mil. Mr. Etheridge mentioned some recent influences on Navy HSI:
- Public Law 107-314, National Defense Appropriations Act 09-07
- GAO Report 02-520, June 03, Navy Actions Needed to Optimize Crew Size and Reduce Total Ownership costs.
- Systems Engineering Acquisition and Personnel Integration Report, 2004.
Some history of Navy investment in HSI:
- IMPRINT adopted – 2005
- MANPRINT adopted – 2005
- IMPRINT Improved – 2004
- Improved job performance – 2004
- Reduced total ownership costs – 2004
- Continue development of Manpower/Personnel tools - 2005-2006
- Develop advanced cognitive evaluation tools – 2006
- NAVPRINT - 2007
- Develop advanced tools and methods – Future
- Human Analysis & Requirements Planning System (HARPS) – 2007-2008 (contains and integrates all HSI analyses)Joint Capabilities Development Integration System (JCIDS)
- HSI guidance (OPNAVINST 5310.23)
- Virtual System HSI Guide- Volumes 1 (overview), 2 (implementation) and 3 (how-to)
According to the National Defense Appropriations Act, the LSI role as we know it today will disappear, with the LSI job going to the service or to a consultant.
The way ahead for the Navy includes the following:
- OPNAVINST 5310.23 implementation
- HARPS functional requirements document
- Implement HARPS and OPNAVINST 5310.23
- Implement FY07-08 funding plan
- Update JCIDDS 3170.01
- Revise SECNAVINST 5000.2D
- Submit Congressional report on HSI by June 2007.
Reducing Risk for a Global C2 Enterprise Collaboration Capability was presented by Mr. Don Monk, Air Force Research Laboratory, Human Effectiveness Directorate, don.monk@wpafb.af.mil. Mr. Monk’s current job is in visualization and collaboration for air operations centers. Collaboration is viewed on a continuum:
Cooperation --------------- Coordination --------------- Collaboration
Short term, informal Long term – around Pervasive &
Parties have different goals a single project Continuing
True collaboration is about providing a common workspace across all participants. The Air Force is currently funding 6.2 and 6.3 projects leading to collaboration technology and fully distributed air operations centers.
Risk Reduction Through Collaboration: Army Views was presented by Dr. Elizabeth Bowman, Army Research Laboratory-Human Research and Engineering Directorate, ebowman@arl.army.mil. Dr . Bowman briefly described two activities underway examining collaboration:
- Soldier-Robotic Systems Teams: Soldier-PakBot teams were investigated. Some of the issues discovered were:
- Sensor camouflage is needed
- Security /soldier survivability problems – Soldier distraction while operating robots
- Use in the Rain proved difficult or impossible
- The Stylus (for interacting with the controller) got lost – Soldiers used knives or whatever was available as a replacement
- Joint/Operational View: Multi-national experiment #4 sponsored by JFCOM.
Collaboration between NASA and the National Space Biomedical Research Institute (NSBRI) was presented by Dr. Edna Fielder, Baylor College of Medicine, efielder@bcm.edu. Dr. Fielder is working on collaborative project with NASA in human performance, design and crew procedures. [also please see http://www.nsbri.org]
HSI Workshop Meeting - Update on HSI/MANPRINT Guidance to Industry Activity. Mr. Adrian Salinas held a short update session to provide feedback to the group on the status of an activity requested by Dr. Robert Foster. Dr. Foster has requested the DoD HFE TAG to provide assistance in providing improved HSI guidance to industry. Mr. Salinas described a one-day workshop held in Orlando in June 2006, the objective of which was to refine a Data Item Description (DID) on Human Systems Integration Program Plan and to identify additional methods of providing guidance to industry. Some of those methods included development of generic HSI briefing charts and development of a MIL-Handbook on HSI.
Mr. Salinas indicated that Acquisition, Technology and Logistics (AT&L) will be getting task to provide DoD guidance to the Services. The TAG’s activities to date are relevant to that work. The TAG needs to provide Dr. Foster with a short description of what has been done thus far. A 5-10 chart briefing is needed, emphasizing:
- Need: It has been >4 years since DoDI 5000.2, Encl (7) described the HSI job.
- Progress: The TAG has helped author a DID and plans to work on a MIL-Handbook and more.
- Status: The DID has been approved, the Mil-Handbook and a DID for HSI Report are underway
- Plan Forward: Continue with the Mil-Handbook, briefings, and training materials and DID for HSI Report.
In addition to the briefing charts, a succinct white paper (1-2
pages) would also be useful.
Sub-Group Meetings Attended at the DOD HFE TAG:
Technical Society/Industry SubTAG. The Technical
Society/Industry (TS/I) Sub TAG met twice during the TAG meeting on Tuesday
morning and afternoon. Dr. Jen
Narkevicius (jnarkevicius@jeniussolutions.com)
chaired both two meetings.
In the morning session,
· Alan Poston (representing HFES Institute) expressed appreciation for TAG inputs on areas where MIL-STD-1472 requires updates.
· DIDs – The Navy selected six HFE DIDs for publication, plus two more (one of which is for the HSI Program Plan. Marcie Langelier is the Navy’s point of contact, who is working on them with Jen Narkevicius.
· Dick Armstrong reported on a Navy project to develop an HSI MIL Handbook. A draft is planned for late in June or early in July 2997. The TAG will review it following completion of the draft. Dick Armstrong can be contacted at dick_armstrong@teambci.com or (850) 231-1155.
In the afternoon session, the first speaker was Stephen C. Merriman (972-705-8219, stephen.c.merriman@boeing.com), who described the HSI/MANPRINT approach being taken on the Army’s Future Combat Systems (FCS) Program. He described the domains included within program MANPRINT, described the 14 systems constituting the FCS family of systems and briefly discussed the larger “system of systems” which includes literally dozens of other existing systems with which FCS must interoperate. ON FCS, the Lead Systems Integrator is responsible for preparing the OSD-mandated Manpower Estimate report as well as the Target Audience Description.. While difficult, these two efforts have provided great insight into the 100+ MOS needed to support the FCS brigade combat teams, the training requirements and the numbers of personnel. This in-depth M&P knowledge provides essential information for making MANPRINT tradeoffs affecting the MP&T domains. Challenges in the HFE domain include assuring commonality, not just at the display or crew station level, but also at the platform and system of systems level. To help assure commonality, the LSI has developed a five-volume Soldier-Machine Interface Standard, applicable to all 20+ FCS suppliers (or “One Team Partners).
The second speaker was Julie Stark, PhD (jstark@systechnologies.com) who presented Human Factors Does Not Equal Human Systems Integration. There are 5major differences between the two terms.
Human
Factors Human
Systems Integration
Discipline / Field of study Approach
Science-based Policy-based
Specific Issues Broad Issues
Focused on User Focus on Trade-offs
What makes an HSI person?
1. Extensive knowledge of one domain that doesn’t interact with other domains?
2. Extensive knowledge in one domain that interacts with other domains?
3. Knowledge and interactions with two or more domains?
4. Knowledge of all domains?
Human/Robotics Interface Special Interest Group (SIG). This SIG met for the first time as a closed government session at the last TAG meeting. It was opened up to all TAG participants at TAG-57.
The first presenter was Dr. Mark Steinberg, Office of Naval Research and Naval Air Systems Command, who spoke on Span of Control. Some of the current issues in robotics include the control of heterogeneous vehicles, different levels and types of autonomy, significant differences in communications (reliability, bandwidth, etc), highly limited manning on ships (low skill, no dedicated UMS controllers), moving to common control systems and interoperability.
Some approaches to span of control include:
· Theoretical approaches and modeling
o Requires simplification
o Difficult to address performance, contingencies, novel systems
o Single operator approaches may not scale to groups
· Empirical studies of complex operational scenarios/systems
o Expensive to do statistically meaningful studies
o Not clear how much results can be generalized
Intelligent Autonomy Program is managed by Dr. Steinberg. This program includes:
· Mission management of 5-10 heterogeneous unmanned systems of 3-5 types with a common control system.
· Highly autonomous re-tasking and fully autonomous
· Multi UxV (variety of unmanned systems)
· Maritime Image understanding
The main theoretical approach with DARPA has been neglect tolerance:
# Robots = 1 + Neglect time (how long robot can operate unattended)
Interaction time (how long operator
needs to attend to the robot)
The test approach to the experiment was:
· limited training time for 7 operators
· UUVs and UAVs
· vehicle operations/sensor operations/mission commanders
Measures used:
· SAGAT for situational awareness
· Mental Models (drawing maps to represent mental images)
· TLX plus Cooper-Harper ratings
· Reaction Time
· Usability scales
· User satisfaction scales
Next steps: experiment with more complex systems. Vary the number and type of vehicles, mission complexity, automation levels and types, use common tests and methods. Future Navy science and technology direction with regard to robotic control:
· Shared/collaborative control
· Shared control by non-dedicated teams
· Impact of cognitive models in autonomous systems
· Shift from management of vehicles to management of missions
· Human directed learning by autonomous systems
· Larger numbers of unmanned vehicles for persistent surveillance of larger, more complex areas
The second presenter was Dr. Michael Patzek (937-255-7883, Michael.patzek@wpafb.af.mil), System Control Interfaces Branch, Warfighter Interface Division, Human Effectiveness Directorate, Air Force Research Laboratory, who spoke on Air Force Unmanned Systems Control Activities. The following appear to be emerging needs, with the vision being “multi-UAV supervision”:
· Rapid response capability for ISR and strike
· Persistence in multiple regions
· Increased span of control with a single control station
The 2003 summer UFAF SAB study concluded with the following needs:
· Reduce costs and accidents
· Increase span of control
· Greater persistence
· Better mission management, vehicle autonomy and human-computer interface
· Better SA and reduced operator workload
· Better measures of performance
The issues include bandwidth and system time delays.
The Vigilant Spirit program grew out of J-UCAS simulation software. Candidate decision support tools include intelligent cueing, playbook delegation, change detection, Information fusion and expectation management. Sensory interface technologies being examined include: Helmet-mounted display, haptic feedback tactile display, speech recognition, synthetic overlays, and symbology improvements. One activity involves Multi-aircraft video-Human/Automation Target Recognition – basically, computer-assisted scheduling of video presentations to human operators from different sensors.
An advanced development program (6.3) involves multi-UAV supervisory control interface technology MUSCIT) for Intelligence, Surveillance, Reconnaissance (ISR) and Close Air Support (CAS). The program focuses on controls and displays, decision aids, multi-UAV. It is sponsored by the Predator program office and others. It is examining adaptive interface using a predator control station, operated by 6 people, two people or one person. The task is to control 1, 2, 3, or 4 UAVs. The program will be funded from FY-07 through FY10.
The program goals are to:
· Reduce operator-to-vehicle rations
· Increase mission effectiveness
· Increase awareness of system state
·
Identify potential upgrades to
existing systems
Human Factors Standardization (HFS) SubTAG: The chair/chair select for the Human Factors Standardization SubTAG is Mr. Dave Britton david.briton@wpafb.af.mil). New incoming co-chairs will be Josh Kennedy and Alan Poston.
· MIL-STD-2525 Update: Jacob Wetzel, PhD reported on activities in support of revision C. The symbology standards committee is scheduled to ratify the new version in June or July 2007.
o The standard will be reorganized to include appendices for the various domains (air/space, land, maritime). Appendices will include command and control symbols and command and control tactical graphics.
o A boundary octagon will be included within each symbol with four different information areas (top, left, right and bottom).
o MIL-STD-2525 will be harmonized with NATO STANAG 2019 (App 6C) for space symbols. Both -2525 and App-6C will adopt white dotted frames to mean “assumed.”
o Purple color will indicate Commercial Aircraft (COMAIR) tracks. This will aid in distinguishing them from military tracks without having to remove them from the display.
o An optional external status bar, located beneath the symbol, will indicate operational status [green=fully operational; yellow=substantially operational; red= marginally operational; back -= non-operational].
o Symbol colors now may be modified only by varying luminance (brightness).
· NASA Standard 3000 Evolution: Barry Tillman reported on efforts underway at NASA to update the standard by dividing it into two parts, standards and an implementation guide. The standard will consist of two volumes: Volume 1 – Medical, and Volume 2 - Environmental Health. Each program (e.g., ISS) will be responsible for its own implementation guide. A Human Integration Design Handbook (HIDH) will include design guidance and provide assistance in developing program specific requirements. It will be a companion to the standards.
· MIL-Handbook for HSI: Dick Armstrong summarized his efforts in drafting a military handbook for Human Systems Integration. He is working on this for the Navy and is coordinating with the DOD HFE TAG.
· MIL-STD-1787: A new section is being added on Rotary Wing symbology.
· HFE/HSI DIDs: Jennifer Narkevicius and Marcie Langelier are proposing to transfer responsibility for the HFE DIDs from the Army to the Navy. HFE DIDs will be transferred and DIDs for HETR and HETP will be added back in. HFAC-81743 will address Human Systems Integration Program Plan and HFAC-81744 will address Human Engineering Program Plan.
· MIL-STD-1472: A proposed notice change to two tables on lighting and system response times will be cancelled. A survey is being conducted to identify areas in need of revision. Responses are being compiled now and should be ready for reporting at the November TAG meeting.
· MIL-STD-1474: Still working on an update to incorporate revisions by Army Research Laboratory.
·
HFES Report: HFES-100 (computer work stations) is in
work HFES-200 is in the comment cycle.
Craft Bio-dynamics Special Interest Group (SIG). This special interest group focuses on human factors engineering issues unique to watercraft. The first presenter was Dr. Ron Peterson, with L3COM (ronaldpeterson@comcast.net), who spoke on Long Term Health Surveillance for Naval Special Warfare High Speed Craft. Special warfare craft pose extreme hazards for occupants. The bio-dynamic environment adversely impacts performance and may cause injury. The long term health surveillance system was created to monitor this environment. This presentation focused on the Mk V SOC special operations craft. This craft has a history of causing problems. The CACTUS motion data acquisition system is being installed on several craft to acquire data. In FY03-04, a suspended seat was investigated for the Mk V SOC craft; four candidates were evaluated and one was confirmed as superior. Other efforts are examining throttle and throttle location, and a semi-active seat. ISO2631 Part5 was selected as the primary criterion for these studies. Special operations craft pose both acute and chronic problems. Stress is cumulative, resulting in lumbar spine problems (material failure). Operators are tagged with RFID to keep track of what craft they ride and when, co cumulative exposure can be tracked. This work is being sponsored by ONR and coordinated with Navy Health Research Center NHRC).
The next presenter was Cameron Bass, University of Virginia and NWSC Dahlgren VA, who spoke on Assessing Risk of Impact Injury in High Speed Craft. NSW Rigid and Mk V SOC are typical high-speed craft. At speed, every 1 to 3 seconds the craft experiences vertical g’s in the 7-10 g range falling over 1 second. Approximately 65% of craft passengers have self-reported injury (89% in the past two years), with 33.6% being lower back injury. ISO2361 Part 5 (3-axis spinal model) was developed for application to Army vehicles such as Stryker (repeated jolts). Modeling is to predict injury from craft accelerations. The model has been validated to +/- 4 gs but doesn’t account for complex postures; it also under-predicts spinal load above 5 gs.
3-axis seat pan à 99 coefficient neural net à Peak lumbar spin acceleration à Injury Assessment
Madymo modeling (Mathematical Dynamic Modeling) was investigated due to under-prediction by the neural net. This model was much better at predicting peak acceleration of the lumbar spine above 4 gs. Current efforts involve incorporating the Madymo model into the surveillance program. Meta-models have been developed to relate impact acceleration to L4 accelerations so they can go from input to seat to L4.
The next presenter was Dr. Mike McCauley, Naval Postgraduate School (831-6556-2191, memccauley@nps.edu). Dr. McCauley and Eric Pierce, M.S., NSWC Panama City, have been investigating High Speed Vessels, Motion and Human Performance. It has been reported that 90% of Utah Marines on the USS Swift, in 8-10 ft. seas @ 40 knots were motion sick. The Littoral Combat Ship may expose passengers to similar environment, causing motion sickness. Past investigations have been conducted on HSV-25 and five other boats. The Benchijigua Express is a 40+ knot high-speed ferry that can carry up to 1300 passengers. NSWC gained permission to instrument the ship and track data for the two-hour transits between LaPalma and Tenerife. Eighty-six transits were recorded and 1994 questionnaires were completed. The Motion Sickness Questionnaire (MSAQ) was used. It was found that there was a higher incidence of motion sickness reported during evening transits (after sundown) than during the day. Beam seas were found to have minimal impact on balance and motion sickness. Still, 60-90% of participants reported some motion sickness, although most motion effects were fairly benign. New data will be obtained from a twin hull ship, the Sea Fighter (FSF-1). The LCS will be tested in 2008.
Modeling and Simulation SubTAG: Mr. Tom Mitchell served as SubTAG chair,
filling in for LT. Grubb who had a conflict. The presenters for this session
were all Cadets from the US Military Academy. The first presenter was Cadet Erik Mancik, who spoke on Modeling
Targeting Performance Under Physiological Stress. It was already known
that three days of sleep deprivation can cause a reduction of 20% on simple
mathematical tests. The research
questions for this effort were:
Can changes in heart rate be used to model a Soldier’s shooting
performance? And, how does sleep level affect precision? An engagement skills trainer was used
to demonstrate shooting skills.
The dependent variables were precision, time to first shot and total
time. It is known that
physiological stamina and heart rate are reasonably good predictors of
precision. Some stimulation is needed to achieve optimal performance while
additional stress will cause a significant decrease.
The second
speaker was Cadet Matt Webb, who spoke on Detecting Change in Social
Networks Using Statistical Process Control.
Network analysis and statistical process control were used to quantify the
connections between people. Network measures included: network density, “Between-ness” (High
would represent a single person linking many others), and Closeness (a measure
of the number of connections to others – affects the speed of getting
messages out). The first
application was to examine density, closeness and Between-ness of the Al Qaewda
network. The second application was to the Tactical Officer’s Education
Program, where Cadet Webb knew all members of the group and laptop computers
were used to capture all interactions between group members. This project ran over two semesters. They used a model to predict
performance in the second semester based on first semester data. There is
potential to use this method to analyze terrorist groups and other types of
groups.
The third
presenter was Cadet Mancik who spoke on Laboratory Simulation of
Hierarchical and Network Based Organizations.
ELICIT (Experimental Laboratory for Investigating Communications and Trust) was
used. Students were presented with
lots of ‘factoids” of information and their jobs were to piece together the
information in order to predict WHAT is going to happen and WHEN – such
as a terrorist attack. Thirty-four cadets were split into two groups of 17 each
– one hierarchically organized group and one self-organizing group. Both groups were briefed on the same
scenario for 20 minutes. All
messages between people had to be passed through “the system” so that they
could be tracked. The Organizational Risk Analyzer (ORA) was used to analyze
network performance. The
self-organizing group was more efficient, quicker in coming to an agreement
(although they were incorrect).
The hierarchically organized group (single platoon lead, four squad
leaders and three in each squad) took longer but the key people in the group
agreed on the correct answer – therefore it was more effective.
The last
presenter was Cadet Patrick Kuiper, who spoke on Analysis of Sensitive
Equipment Decontamination (SED). The military currently has no means to
decontaminate sensitive equipment.
JPEO-CB is evaluating several alternatives. There are procedures for decontaminating soldiers and large
equipment, but not laptops, PDAs, etc.
The US Military Academy evaluated alternatives using mathematical
modeling of how long different techniques would take to decontaminate the
sensitive equipment. A PRO-MODEL
simulation (discrete event simulation) was set up to compare the liquid
immersion (fast but can only accommodate small
amounts of equipment at one time) or vapor-based (hydrogen peroxide + ammonia) techniques. USMA took action to look
into speeding up the vapor-based method using mathematical modeling.
User-Computer Interface SubTAG: The Sub TAG meeting is co-chaired by Nausheen Momen, LT, MSC, USNR, Ph.D. (nmomen@namrl.navy.mil) and Stephen Merriman (stephen.c.merriman@boeing.com. There were 31 attendees at the session, on Tuesday, 7 November 2006. No changes were proposed in SubTAG leadership at this meeting. No revisions were proposed to the UCI SubTAG. Four presentations were made at this session.
The first presenter was Cadet David Dawes
(USMA), who spoke on Scaled Displays in Language Translation. There are currently
two main methods used to translate foreign intelligence documents into English
for analysis: Linguists and
ARL-developed FALCON (Forward Area Language Converter)
that is hosted on a laptop computer.
Linguists are slow but highly trusted. FALCON is faster but does not produce reliably-correct
translations. The USMA investigated
placing an improved version of FALCON onto a small Personal Digital Assistant
(PDA) that would be highly portable. The Language Instant Screening Tool (LIST) was
developed for this purpose. It is equipped with a small camera to capture
images of document pages. It uses optical character reading (OCR) to translate
the images into text and then performs machine translation, semantic text
analysis and other processing to derive thematic intent. The Cadets compared
LIST performance against linguist translators. They also compared LIST to the
customer’s needs (simple, fast, survivable, durable, portable, little training,
long battery life, maintainable and accurate). They build a value model to determine where LIST would be
most valuable. Using their value model, LIST scored an 80 while linguists
scored 50. A sensitivity analysis was conducted and it was determined that LIST
is superior EXCEPT when battery life is low AND weight is high. LIST battery life is currently 4.5
hours and weight is minimal. The use
of LIST would create a need for doctrine change wherein LIST would be used to
perform initial triage on documents to quickly determine which are of high
value and therefore should be quickly translated and provided to the S2 for
action. In order to define follow-on work, USMA would like to have a point of
contact at SOCOM, who may be interested in pursuing this area. Next actions
planned include investigation of:
·
LIST output – what should it
say?
·
Product durability requirements
·
Desirability of connectivity via a
network
The second presenter was Mr. Mike
Hűbler, US Army RDEC Software Engineering Directorate (mike.hubler@us.army.mil), who spoke
on Toward a Common Health and Status Panel for Missile Systems. The Common Warfighter
Machine Interface (CWMI) team, as the government voice for all user interface
(UI) issues in the Integrated Army Air and Missile Defense (IAMD) community,
was tasked to design a standard UI adapted to any user performing any task
within an integrated system of AMD systems. CWMI has distilled AMD user
interfaces into individual, general CWMI components that can be assembled and
customized based on user roles. This presentation focused on the design of a
“health and status” component that represents the condition of sensors,
launchers, communications and other equipment in any AMD system. The need for a
universal component to handle health and status display, as well as the
functional description of the component is based on extensive research of
display requirements and screenshots from systems such as Patriot, THAAD,
Avenger/SHORAD, NASAMS, and GMD.
Recommendations for CWMI component designs
stem from the analysis of quantitative and qualitative data generated by a
series of usability tests. The CWMI team conducted a focus group with twelve
AMD warrant officers at Fort Bliss, TX to help differentiate the display needs
of different echelons and roles. CWMI developers rapidly produced three fully
interactive JAVA prototypes, based on military and video game industry design
research, for testing in two experiments. Using a prototype test engine, the
team collected logs of detailed mouse event and timing metrics on information
retrieval and action scenario tests from 34 AMD subjects. Each experiment
includes a detailed interview and the unique ability to replay user interaction
using the logs. These qualitative tools provide a strong interpretative lens
for viewing the quantitative performance results, allowing usability engineers
to not only judge a “winning” prototype, but to modify the winner based on more
nuanced understandings of what worked well from each of the three prototypes.
The third presenter was Chris Carobine, NETWARCOM,
who spoke on It's all about the NEWS. This talk included an
overview of the user centered design considerations during the development of
the NETWARCOM Enterprise Work Space (NEWS) project, including a description of
the early and continual SME involvement, Six Sigma methodologies,
documentation, and ongoing training efforts. The phased implementation of NEWS was also be delineated.
The fourth presenter was Mr. Mike Hűbler, Army RDEC Software Engineering Directorate, (mike.hubler@us.army.mil), who spoke on Collaborative user-interface development: How a CORUS might review ACTORs on a STAGE. Human Machine Interface (HMI) development, with its unique emphasis on human usability, graphic design and communication, should be its own creative effort, not fused to the same software engineering teams or processes that produce the internal algorithms, information management schemas and other back-end functions of an application. The RDEC is developing a collaborative environment for HMI innovation by separating user interface development from the rest of the application and then distilling HMI’s further into components. Prototypes of HMI components could be individually evaluated and peer-reviewed with a test application that stimulates them with data. The full capability environment for HMI component development can be accomplished with three elements: the standard GUI evaluator (STAGE), adapted HMI components to review (ACTORs), and a collaborative online review and update space (CORUS).
Human Factors Engineering/Human Systems Integration: Management and Applications. The co-chairs for this SubTAG are Mr. Brad Collie, (Bradley.collie@navy.mil) and Daniel Wallace (Daniel.wallace@navy.mil). The first presenter was Michael Salyer, Northrop Grumman Corp, who spoke on A Review of Air Force and Joint Forces Capabilities Documents for HSI Requirements and Concerns. JCDs and ICDs were reviewed for HSI requirements. CDDs and CPDs were also reviewed (sections 6 and 14. It was determined that there are five levels of requirements:
- Clearly stated, verifiable, objective and threshold values, measurement criteria.
- Clearly stated, no measurable criteria
- Not clearly stated, cannot be verified, with rationale for HSI concept
- Not clearly stated, cannot be verified, with no rationale for HSI concept
- Not HSI.
The document most commonly containing HSI requirements was the CDD (53%of all HSI requirements). It was determined that 90% of all HSI requirements were not verifiable. Within HSI, 61% of the HSI requirements were in the Human Factors Engineering (HFE) domain. Eleven percent of all HSI requirements were KPPS (5%) or KSAs (6%). There was a very high level of variability in terms of type; requirements were generally poorly stated. There is a huge room for improvement.
The second presenter was Bonnie Novak, Serco, Inc. (bonnie.novak@serco-na.com), who spoke on HSI Port: Current Capabilities & Future Enhancements. PORT is a Program On-line Review Tool. HSI Subject Matter Experts (SMI) determined what HSI topics should be addressed at program reviews. HSI PORT was launched two years ago to expedite HSI reviews. It is a common web-based forum, with standardized HSI questions for all HSI domains. There are two main functions:
- Information Source: HSI PORT hosts 90 tools, metrics, DOD policies and points of contact.
- Review Source: Accesses program documentation and databases of questions used to assess programs.
HSI training courses are listed on the website: http://www.hsiport.serco-na.com. Contents include:
- Standard checklists
- Program documents
- HSI Assessment Reports
- Tools, metrics and guidance
- Password protection
After the rating is completed, the web site integrates the data and lists out: strengths, opportunities for improvement. In the future, activities will concentrate on:
- HSI tutorial
- Links to reports
- Weighted risk assessment matrix
- Lessons Learned
- ROI statistics
- On-line chat forum to facilitate HSI decisions
For further
information, contact Bonnie Novak or Bob Bost (bob.bost@serco-na.com).
The next presenter was Dr. Ken LaSala, KPL Systems (kplsys@verison.net) who spoke on Design Collaboration Through REHMS-D (Reliable Human Machine Systems Developer). REHMS-D follows the Systems Engineering process. It uses reliability as a metric for selection of human interface ant task parameters.
The next presenter was Dr. Dale Nute, FSC Panama City, FL (dnute@pc.fsu.edu), who spoke on Underwater Crime scene Investigation. He provided some insight into some of the unique requirements and experiences involved with underwater crime scene investigation. FSU began the program at about the time of the USS Cole attack The current approach is to pull up evidence from underwater and examine it at the surface. But, patterns underwater can reveal evidence that can be missed. Customers for underwater crime scene investigation include insurance companies, bridge/dam inspectors, environmental investigators. Underwater crime scene investigators are part investigator, part scientist and part diver. Some of the tasks performed include location and recognition of potential evidence, measurement and documentation of the scene and evidence, collection and packaging of evidence, examination and interpretation of evidence at the scene. Areas of difference between “normal” investigation and underwater investigation fall into four categories: time limitations, sensory limitations, dexterity issues and hazards. New tools are always being sought to help see underwater, measure underwater, document underwater and survive underwater. Robots and side-looking sonar, interfaced to a network, would allow remote operations. Some of the detection equipment used include: eyes, binoculars, photographs, thermal, remote operations vehicles, canines, magnetometers, metal detectors, ground penetrating radar, sonar and chemical detectors.
Design Tools and Techniques SubTAG: Did not meet at
TAG-57.
Workload and Stress SubTAG. Did not meet at TAG-57
Personnel Selection and Classification SubTAG: Not attended.
Sustained/Continuous Operations (SUSOPS/CONOPS) SubTAG: Not attended.
Human Factors Test and Evaluation
SubTAG: Not attended.
Human Factors in Training SubTAG: Not attended.
Controls and Displays SubTAG: Not attended.
System Safety/Health Hazards/Survivability SubTAG. Not attended.
DOD HFE TAG Operating Board Meeting:
1. TAG Reports: SubTAG chairs reported
the numbers of presentations, charter changes and other items of interest.
2. Service Reports:
Army: Suggested holding some TAGs where there
may not be a strong military presence but there is a strong industry presence
Navy: Brad Collie is the new Service
representative. The Navy requests
that all papers and abstracts be placed on the TAG website and that the TAG
theme be posted earlier to facilitate paper preparation.
Air Force: Adrian Salinas will become the USAF Service representative
in May 08. TAG-59 may be either at Hurlbert Field or Eglin AFB. The theme will
be either best practices or unmanned systems. The USAF would like to see more
emphasis placed papers emphasizing scientific method.
3. Call for Papers: Future calls for papers will be linked to the web page. Sheryl will
investigate electronic submission.
4. Panel Facilitation: SubTAG chairs need to
ensure that all panelists are in attendance and that there is an equal division
of time available for each panelist.
5.
Dr. Foster Initiative: A white
paper and accompanying charts will be prepared within the next 30 days on the
“HSI Guidance to Industry” issue.
The topics to be covered are need, progress, status and recommendations. When complete, they will be sent
to the executive committee for action.
Steve Merriman will prepare these documents.
6.
Operating Structure Changes: A
discussion was held on changing the orientation of the TAG from HFE to
HSI. While the Air force advocated
this change, the Navy and Army preferred staying with HFE as the primary focus
of the TAG. HFE is more aligned with science and engineering, while HSI is more
of a management model. A vote was taken and it was confirmed that the TAG would
remain aligned primarily aligned with HFE.
7.
Next TAG meeting: TAG-58 will emphasize
extreme human factors engineering - Novel applications.
Submitted by:
Stephen C.
Merriman
DoD HFE TAG, TS/I
Credentialed EIA, SAFE, AsMA Representative
FCS MANPRINT Sr.
Manager
3001 E. George
Bush Highway, Suite 225
Richardson,
TX 75082-2777
972-705-8219
(Office)
214-316-7071
(Cellular)
972-705-8098
(FAX)
scmerriman@tx.rr.com (Home)
ATTACHMENT (1)
DOD HFE TAG Background
The DoD HFE TAG was begun via memorandum of agreement signed by the Service Secretaries in November 1976. Goals of the TAG were established as follows:
• Provide a mechanism for exchange of technical information in the development and application of human factors engineering.
• Enhance working level coordination among Government agencies involved in HFE technology research, development and application.
• Identify human factors engineering technical issues and technology gaps.
• Encourage and sponsor in-depth technical interaction, including subgroups as required in selected topical areas.
• Assist as required in the preparation and coordination of Tri-Service documents such as Technology Coordinating Papers and Topical Reviews.
The TAG addresses research and technologies designed to impact man-machine system development and operation throughout the complete system life cycle. Topics include:
• Procedures for use by HFE specialists, system analysts and design engineers in providing HFE support during system development and modification
• Methodologies to identify and solve operator/maintainer problems related to equipment design, operation and cost/effectiveness
• Mechanisms for applying HFE technologies, including formal and informal approaches to validation and implementation, and the determination of time windows for application.
The TAG comprises technical representatives from Government agencies with research and development responsibilities in the topical areas mentioned above. Additional representatives from activities with allied interests affiliate with the TAG as appropriate. Technical experts in special topic areas may augment attendance at specific meetings. Also participating in the TAG are official representatives of technical societies (e.g., Human Factors and Ergonomics Society, SAFE Association) and industrial associations (e.g., Electronics Industry Alliance) with a stated interest in HFE. These representatives may attend subgroup and general plenary sessions and they must be credentialed by the TAG prior to attending any meetings.
To facilitate detailed technical information exchange, the TAG is composed of committees and subgroups, or “Sub TAGs.” Committees are established to address specific issues or problems and are disestablished upon completion of their tasks. Sub TAGs address problems of a general or continuing nature within a specific field of HFE technology. Membership in Sub TAGs and committees may include non-government personnel involved in research, development and application. Attendance by non-government individuals is possible if the person is either sponsored by a government agency or if accepted by the TAG chair prior to the meeting. Chairing of the various subgroups and committees is typically rotated among the Services and in some cases, NASA, as provided in individual charters.
The current sub-groups typically meeting at the HFE TAG meeting were as follows.
Sub-TAGs:
- Controls and Displays
- Design: Tools and Techniques
- HFE/Human Systems Integration: Management and
Applications
- Human Factors in Extreme Environments
- Human Factors and Operational Medicine
- Human Factors in Training
- Human Factors Standardization
- Human Factors Test and Evaluation
- Human Modeling and Simulation
- Personnel Selection and Classification
- Sustained/Continuous Operations Core Competencies
- System Safety/Health Hazards/Survivability
Core Competencies
- Technical Society/Industry
- User-Computer Interaction
- Workload and Stress
Affiliated Groups:
·
Mission
Centric Human Performance Measurement Interest Group
·
Human Robotic Interaction
Interfaces Special Interest Group (New)
·
Craft Bio-dynamics Special Interest
Group (New)
ATTACHMENT (2) TAG-57
Meeting Theme
Reducing Risk through Collaboration
Government acquisition and research
processes are often characterized as highly formalized and complex with
multiple stakeholders who have competing interests. When a research or
acquisition program encounters technological challenges that may increase program
risk via cost overruns, slipped schedules, or poor performance outcomes,
hastily improvised solutions have been met with varying degrees of success.
Technology woes often filter down to the human-machine interface and become an
additional burden to the user. This meeting of the DoD HFE TAG will focus on
efforts to reduce risk through collaboration for government-procured systems
and government-funded research programs.
Effective collaboration has been credited
with lowering costs, reducing development time, accelerating information
exchange, reducing unnecessary redundancy, improving usability, and driving
innovation. Human Factors engineers and Human Systems Integration (HSI) practitioners
can participate in multiple collaborative efforts at different stages in the development
cycle to reduce risk. For example, they can be members of (1) a multidisciplinary
acquisition team in which they work engineering design issues, (2) a joint services
HSI team working AIRPRINT, MANPRINT, and SEAPRINT issues, and/or (3) research
programs with academia and industry partners.
The goal of TAG-57 is to provide a forum
for participants from the military and other government agencies to discuss HSI
within the context of collaboration, cooperation, and partnerships that have
reduced risk or have the potential to do so. Suggested topic areas include:
• Building partnerships
• Case studies, success stories, and lessons
learned
• Developing effective strategies
• Facilitating collaboration, communication,
and information sharing
• Innovative and best practices
• Leveraging resources
• Managing complexity and risk
• Overcoming obstacles
• Sharing tools, methods, laboratories, and
technologies
• Technology transfer
ATTACHMENT
(3)
PROGRAM SUMMARY
Monday 14 May 2007
0830 - 1000 Executive
Committee meeting
1000 - 1100 New member
orientation
1100 - 1300 Luncheon
Break
1300 - 1700 Plenary
Session
1800 - 2000 TAG Mixer
Tuesday 15 May 2007
0730 - 0830 Technical
Society/Industry
0830 - 1100 Human Factors
Test and Evaluation
0830 - 1100 Personnel
Screening and Selection
0830 - 1100 Human
Robotics Interface/Interaction Special Interest Group
0930 - 1000 Networking,
coffee
1100 - 1230 Luncheon
Break
1230 - 1430 Human Factors
Standardization
1230 - 1430 Controls and
Displays
1230 - 1700 Human Factors
in Training
1430 - 1500 Networking,
coffee
1500 - 1700 Human Factors
in Extreme Environments
1500 - 1700 Craft
Biodynamics Special Interest Group
1700 - 1830 TS/I and
Service Caucuses
Wednesday 16 May
2007
0730 - 0830 Distributed
and Dismounted Warfare Special Interest Group
0830 - 1100 Human
Modeling and Simulation
0830 - 1100 Workload and
Stress
0930 - 1000 Networking,
coffee
1100 - 1230 Luncheon
Break
1230 - 1430 Design: Tools
and Techniques
1230 - 1430 User-Computer
Interaction
1430 - 1500 Networking,
coffee
1500 - 1700 Human Factors
Engineering/Human Systems Integration: Management and
Applications
1500 - 1700 System
Safety/Health Hazards/Survivability
1800 - 2200 Social
Thursday 17 May
2007
0800 - 0930 Operating
Board
1015 - 1700 Tour
(unofficial times – brown bag lunch available for purchase)
Note – The Workload
and Stress, Design: Tools and Techniques, and the Sustained/Continuous
Operations subTAGs will
not meet at TAG-57.
ATTACHMENT (4) DoD HFE TAG Operating Board
|
|
|
|
|
|
|
Proponent |
|
|
|
Current
Chair (Air Force) |
|
|
|
Vice Chair
(Army) |
|
|
|
Immediate
Past Chair (Navy) |
|
|
|
Army
Representative |
|
|
|
Navy
Representative |
|
|
|
Air
Force Representative |
|
|
|
FAA
Representative |
|
|
|
NASA
Representative |
|
|
|
Program
Coordinator |
|
|
|
|
|
|
Co-Chairs:
Ms. Marianne Paulsen
|
|
|
Co-Chairs:
Dr. Pamela Savage-Knepshield
|
|
|
|
Human Factors Engineering/Human
Systems Integration: Management and Applications Co-Chairs:
Mr. Brad Collie
|
|
|
|
Human Factors in Extreme
Environments |
|
|
Co-Chairs:
Dr. John Stewart
|
|
|
|
Human Factors Standardization |
|
|
|
Human Factors Test and Evaluation Co-Chairs:
Mr. Darren Cole
|
|
|
|
Human Modeling and Simulation |
|
|
|
Personnel Selection and
Classification Co-Chairs:
LT Tatana Olson, Ph.D., MSC, USNR
|
|
|
|
Sustained/Continuous Operations Co-Chairs:
LCDR Walter Carr
|
|
|
|
System Safety/Health
Hazards/Survivability Co-Chairs:
George Murnyak
|
|
|
|
Technical Society/Industry |
|
|
Co-Chairs:
LT Nausheen Momen
|
|
|
|
Workload and Stress |
|
|
|
|
|
|
|
|
|
Mission Centric
Human Performance Measurement Interest Group |
ATTACHMENT (5) DoD HFE TAG Attendees
ATTACHMENT (6) DoD HFE TAG Policies
1. Membership
(General membership policies are outlined in the Operating Structure, under
"Group Composition.")
1.1 Individuals who are not affiliated with
Government agencies (but who are associated with technical societies or
industrial associations with a stated interest in human factors engineering)
wishing to affiliate with the TAG may contact the current Technical
Society/Industry SubTAG Chair to ascertain eligibility under the TAG Operating
Structure. Once eligibility has
been ascertained, the individual should submit a letter on the organization's
letterhead, confirming his/her status as the organization's representative, to
the current Chair of the Technical Society/Industry SubTAG.
1.2 Emeritus Membership may be approved by the
Executive Committee on a case-by-case basis for a former TAG member who is
retired from government service or defense industry. Emeritus Membership is automatically deactivated during any
period or re-employment with the government or defense industry.
2. Meeting
Sites (Sites are recommended by the service caucus whose turn it is to host
the TAG with a view toward a balance in geographic location and meeting
facilities.)
2.1 TAG members are encouraged to recommend
potential meeting sites.
2.2 Organizations who wish to host the TAG should
contact their Service Representative or the current TAG Chair.
3. Agenda
(The agenda is determined approximately three months before the scheduled
meeting. The Chair Select selects
the topics from those recommended by the Service Representatives, hosting
agency and the TAG Coordinator.)
3.1 TAG members are encouraged to suggest potential
agenda topics or topics suitable for tutorial sessions to their Service
Representative, the current TAG Chair, or the TAG Coordinator.
4. Registration
(Registration fees and the date of the close of registration are announced
in an information letter sent approximately two months before the scheduled
meeting.)
4.1 All attendees are expected to pre-register and
prepay by the announced close of registration.
4.2 Only individuals receiving late travel
approvals may pre-register on-site.
Payments made at the meeting site must be in cash.
5. Minutes (The Minutes of each meeting serve as
the principal mechanism for the reporting of TAG activities. The Minutes will be published as a
draft document on the website.)
5.1 Individuals or agencies desiring to be included
on the distribution list for a specific meeting should contact the TAG
Coordinator.
6. SubTAGs
and Committees (See the Operating
Structure, section entitled "TAG SubTAGs," for specific information
regarding the purposes and operating procedures of SubTAGs and committees.)
6.1 All SubTAGs and committees are encouraged to
meet in conjunction with the TAG at least once each calendar year.
6.2 All SubTAGs and committees meeting in
conjunction with the TAG are required to provide a chairperson for the specific
meeting.
6.3 All SubTAG and committee chairpersons are to
submit a brief report of each meeting to be included in the set of TAG Minutes
covering the SubTAG/committee meeting time frame.
6.4 All SubTAGs and committees are required to
provide the TAG Coordinator with an up-to-date list of their membership for use
in the distribution of TAG announcements.
6.5 All SubTAGs are required to submit to the
Executive Committee a Charter including, but not limited to, statements
regarding:
· objectives · membership policies · meeting schedule
· scope · chair selection/tenure
6.6 Committees are required to submit to the
Executive Committee a document including, but not limited to, brief statements
regarding:
· objectives
· membership policies
· chair selection/tenure
6.7 Rotation of the chair position is determined by
SubTAG charter. If the position
cannot be filled by the appropriate service at the election meeting, the SubTAG
may progress to the next service willing to chair the SubTAG
7. SubTAG
Establishment
7.1
Groups interested in addressing technical areas not covered by existing SubTAGs
may request the TAG Chair to provide meeting time.
7.2 Formal SubTAGs and committees may be
established by recommendation of the Executive Committee.
8. Chair/Representative
Selection (General
selection procedures are outlined in the Operating Structure under
"Conduct of Business.")
8.1
A Service
caucus may be called by the TAG Chair or the current Service Representative.
8.2 Methods of determining the Chair Select and
Service Representatives are Service dependent.
8.3 Unexpired terms of office will be filled by
appointment by the Executive Committee, until a caucus of the Service can be
called at the next regularly scheduled TAG meeting.
9. Funding The funding required for the
organization, conduct, franking, and documentation of all TAG meetings shall be
done jointly by the three Services and other selected agencies. The specific mechanisms to obtain and
allocate funding from the Services/agencies shall be arranged by the Current
Chair, Chair Select, and Immediate Past Chair.
10.
Policy
Changes
10.1 Additions to or amendments of the above policies
may be recommended by submitting the suggested change(s) in writing to the TAG
Chair.
10.2
Policies may be amended by a majority vote of those Operating Board members in
attendance at the Operating Board meeting at which amendments have been proposed.
Amended 14 November 1989 at TG-23, Killeen,
Texas.
Amended 3 May 1994 at TAG-32, Oklahoma
City, Oklahoma.
Amended 8 May 1996 at TAG-36, Houston,
Texas.
Amended 7
November 2002 at TAG-48, Alexandria, Virginia.