CSU Live/Web Reference Pilot Project, Detail for Product/Service Protocols    
Committee members are encouraged to solicit the input of colleagues and staff when filling out the protocol review form.  Please review element legends to assist with ranking.  Rank in whole numbers only.    
Product under review:   Campus:    
Reviewer's Name:   Dates of Review:    
 Reviewer's E-mail Address:    
   
Product Evaluation    
Please enter a rank (from 1-5, 5 being highest) for each of the following product elements 1=Very Low 2=Low 3=Average 4=High 5=Very High Comments:  
Technical   Ranking          
Software stability[1]          
Software speed[2]          
Ease of access from on campus [3]          
Ease of access from off campus [4]          
Browser and operating system concerns[5]          
Connectivity [6]          
Compatability with campus databases[7]          
Compatability with WebCT, Blackboard, and other instructional applications[8]          
Compatability with Pharos[9]          
ADA compliance[10]          
         
Please enter a rank (from 1-5, 5 being highest) for each of the following product elements 1=Poor 2=Fair 3=Average 4=Good 5=Excellent Comments:  
Service/Support   Ranking    
Vendor responsiveness [11]        
Vendor training[12]      
Quality of documentation [13]          
Amount of support required of front-line library staff[14]          
Amount of support required of technology staff [15]          
Amount of support required of the vendor [16]          
Customization[17]          
           
Please enter a rank (from 1-5, 5 being highest) for each of the following product elements 1=Poor 2=Fair 3=Average 4=Good 5=Excellent Comments:  
Features   Ranking      
Reports and statistics[18]          
Usefulness of product features[19]          
         
Total   0      
           
[20]          
[21]          
[1]
Consider whether your browser,the software, or the computer crashed during use.  Avoid ranking based on campus-specific computing or network problems.
[2]
Consider whether the software functioned at the speed necessary to be useful.  Avoid ranking based on campus-specific computing or network problems.
 
[3]
Rate on accessibility of vendor-provided service only.  Avoid ranking according to problems or issues that arise due to campus-specific computing or network problems. 
 
[4]
Rate on accessibility of vendor-provided service only.  Avoid ranking according to problems or issues that arise due to campus and/or user-specific computing or network problems.
 
[5]
Does the tool work equally well with Internet Explorer and Netscape?  All versions?  Can the tool be used with various Operating Systems? 
[6]
Consider the amount of database downtime;  hours of service, etc.  Do not rate based on campus-specific network, computer, or connectivity problems or issues.
[7]
Consider whether the software allows use of campus databases and the ease of doing so.
 
[8]
Consider possibilities for compatability with campus instructional software. 
 
[9]
Consider possibilities for compatability with Pharos and any issues that might arise.
 
[10]
Consider degree to which the database is useable by disabled patrons, staff, faculty, etc. and to which it complies with Section 508 guidelines.
[11]
Consider the ease of contacting a vendor, the hours that service representatives are available, and the quality of help that the vendor representative was able to give you.
[12]
Consider the quality and quantity of vendor training. 
 
[13]
Consider whether online help is available and its usefulness; instructions that are clear and easy to follow; "plain English" or technical jargon; print documentation, etc.
 
[14]
Is the end-user product reasonably intuitive?  How much support and training will be necessary for people to use it?  Are there confusing features?  Is the interface good?
[15]
Will product integration take a reasonable amount of staff support?  Are there possible time drains involved? 
[16]
Consider number of calls made to vendor technical support.  How often is vendor intervention necessary?
[17]
Consider customization options for the search interface; usage statistics; administrator's options, etc.
 
[18]
Consider whether the product provides the necessary tools to provide useful statistics.  How much can the statistics be manipulated?  Are regular and useful administrative reports generated?
 
[19]
Consider the product features such as transcripts, bookmarks, script templates, etc.  Are these features useful and robust?  Are there important features missing?  Excess features that make the tool too confusing?
 
[20]
Assess cost relative to value. Price alone does not necessarily mean a high or low ranking--evaluate in terms of content, user interface and value added features.   Consider flexibility in pricing options.  Is cost reasonable compared to similar products?   Use comments section if necessary to elaborate.
[21]
Consider any hidden costs such as charges for support.