︎



Improving the usability of the Norman Sicily Project Website for history buffs  


UX Research, Usability Testing



Client
The Norman Sicily Project

Research Methodology
Moderated User Testing

My responsibilities
  • Recruit participants and conducting 2 moderated user tests
  • Analyze and synthesizing data
  • Deliver a series of recommendations for usability fixes across relevant pages
  • Produce mockups of recommendation 4
  • Document part of methodology, findings and recommendations in a usability report
  • Present final findings and recommendations to the client
Duration
March – May 2023 (7 weeks)


Team
Alicia, Karan, Keyi, Yiying


Tools
Figma, Google Docs, Google form, Google Spreadsheet, Zoom




Overview


The Norman Sicily Project Website digitally registers, maps and analyzes the monuments erected during the Sicily Island’s Norman period (ca. 1061-1194), arguably the most auspicious years in its long history. It presents a new model for digital conservation of cultural heritage.

We conducted eight remote moderated usability tests with the target audience members of the Norman Sicily Project website: Scholars, Students, and people with a personal interest in the subject. We carefully matched user goals with pages of the site and found that there is room for significant improvements to usability on important pages to the site. All issues found relate to overall navigation and special site features.

Summary of Recommendations





Introduction to the Norman Sicily Project and
Their User Testing Needs


The Norman Sicily Project is a web app with multiple audiences (scholars as well as educators and the general public), now in a fairly developed prototype form, that promotes historical and cultural heritage knowledge for the island of Sicily during the Norman period (ca. 1061-1194). Yet, though it focuses on Sicily, the app itself provides a model for scholars working on other chronological periods and places - especially in those that are economically underprivileged and do not have the resources for significant cultural conservation efforts.

The site stands out by offering open access to raw datasets and providing interpretations in various formats. The site caters to scholars, researchers, students, and the general public, each with different goals and expectations. Scholars utilize the website to gain fresh insights into society, while teachers incorporate the information into their classrooms. Additionally, the website provides accessible information for the general public interested in learning about Sicily history.

The Pratt Center for Digital Experiences was engaged by the site creators to address their specific need of measuring the usability of the website and effectively conveying their message to different audiences. The goal was to  (1) make the site user friendly, especially for the "Places" and "People" pages;  (2) reorganize it in a way that makes it immediately clear that it serves multiple audiences (scholarly and general public, including educators). To tackle this challenge, conducting moderated usability testing of the site was identified as a valuable approach to gain insights.





Process






Preparation


To kick-off the project, we first met with the client to discuss their concerns and goals, and established the scope of our testing and our user profile. We then prepared a detailed plan and script for testing, recruited participants that fit our user profile with screening questionnaire. We also prepared materials for the testing session including Consent form, moderator script and Pre and Post-test questionnaires.

We designed the following scenario and tasks to explore the core features the client would like to get feedback from, especially for “PEOPLE” and “PLACES” pages, and conducted pilot tests to improve defined tasks.

Scenario


Imagine you are writing a research paper about Norman Sicily. You want to gain a deeper understanding of the Monasteries of Norman Sicily and their Greater Network, as well as the Norman Rulers of Sicily and their Family Networks from the website http://normansicily.org/en/home/.


Tasks

  • Task1: You wish to know the name of the Mother and the Children of King Roger I of Hauteville.

  • Task2: You are searching for Saint Mary of Novara, a medieval church that was acquired by the Benedictine community. Your goal is to obtain the precise latitude and longitude coordinates of the church's location.

  • Task3: Please go back to the homepage. Imagine you are interested in pictures of Churches around Sicily and want to download a photo of a specific one called “Chiesa Bizantina” for your research. Please show me how you would go about this.




Participants Recruitment 


Based on the target user profiles, we started recruiting participants by sending emails to our contacts studying history, scholars related to Sicilian/Italian history and from arbasicula.org. We also posted in Reddit communities incuding r/sicily, r/Italianhistory, r/italianamerican, r/WhiteLotusHBO to find general public with interest in this subject. After three weeks of rigorous recruiting efforts, our team had eight participants. The participants each fit into a key user group.


To better manage each participants’ reponses, we use a shared speassheet with checkboxes to track the progress.



Challege

At first, we reached out to some people listed on the Norman Sicily Project's Website "Credit" section, to invite them to participate in our user testing study because they fit the profile academic professionals. However, after the client received an email from one of them asking about the invitation, she expressed her concerns about this outreach, citing discomfort that the board members may feel due to our approach.
Then we realized we were reaching out the the board members who may be too familiar with the site, and likely have their own preconceived opinions about what changes should be made. So, they aren't best to provide unbiased feedback via user testing and therefore are not ideal participants for our research.
We reply to the client telling her what happened and expressed what our intentions were, we'd follow her lead on what to do next so as to not stir up any further confusion. Although she then gave us the green light to interview three board members who responded, we decided not to include them in our research.



Demographics of Participants





User Testing


We scheduled each moderated user testing session using Calendly. For each session we scheduled one team member to be the moderator, guiding the testing process and recording the session through Zoom. Each session lasted around 30 minutes.

We created poll in Zoom for post-task questions, and launch them before finishing each tasks session to learn about participants’ experience with each user flow.

At the end of all the tasks, we asked a few more post-test questions pertaining to the entire site:

  • Could you tell me about your overall impression of the site?
  • Were the images and graphics on the website helpful in understanding the content?
  • Were the text and font on the website easy to read?
  • Were the interactive features (if any) easy to use and understand?
  • What improvements would you suggest for the website?




Findings & Recommendations


Analysis

We used digital grouping method to compile and analyze our findings in a rainbowsheet. By inputting all the encountered into a spreadsheet, categorized them by where the issue occurred and the nature of them. This grouped all our problems, allowing us to see which ones were experienced by the most participants. We wound up with 4 recommendations towards 4 main issues.




Problems & Recommendations

Participants overall praised the visual appeal and functionality of The Norman Sicily Project website, with comments about the good use of the interactive map featured on the places page, the color, picture and video on the home page. However, our test participants had trouble with some navigation and execution issues when it came to locating and understanding certain information about people and places.
In terms of task completion, 4/8 participants completed the first task1, 4/8 completed the second task, and none were able to complete the third task.

Based on this feedback, we identified the most important areas that require attention. These include the graph describing the greater family of the kings of Sicily, the map providing information on Monasteries of Norman Sicily, the accessibility of the image server link, and the readability and usability of information on the homepage. Our presented recommendations aim to provide practical solutions to the issues encountered during the user testing, thus enhancing the overall usability and user experience of the website.

Our top four recommendations are expanded upon below:
Problem Statement 1: "People" page’s interactive map clutters and lacks of interactivity, leading to difficulty finding information and understanding the legend.


“I found the text to be very small and cluttered. It made it difficult to navigate the graph due to lots of information.”
“I expected the layout functions to be different on the graph. I didn't notice the legend at the bottom corner.”



We received some feedback from participants regarding the graph's clutter and amount of text (Figure A). Participants noted that the legend, which represents the family network links of the Scillians, needed to be discovered and interactive (Figure A). Some concerns were raised about the graph's interaction. Participants had difficulty with the nodes' complexity and functions, such as the primary node losing attention when the participant moved the cursor to another location on the graph (Figure B). While most participants were able to obtain results using the search bar, they found navigation challenging, making it difficult to conduct an accurate search.




Recommendation 1: Optimizing Graph Navigation and Search with Improved Layout and Interactivity
To increase visibility, the search bar should be placed at the top of the page instead of integrating it with the legend (Figure C). Additionally, We have redesigned the Legend for better visibility. We have also added legend connections as filters to enhance the search's specificity.
Furthermore, when a user selects a particular node from the graph to view the family network of a specific king, the node will remain active even if the user moves the cursor. The selection of the node will grey out all other nodes. 





Problem Statement 2:  Challenges with the search function on the "Place" page

“I had trouble finding the right church based on color because I missed the legend in the bottom corner. ”
“I couldn’t retrieve the coordinates of the church because It wasn’t there with other information of the church. I don’t know if search is working or not.“



Participants faced challenges with the search function on the "Place" page of the website, including unclear search results, no visible search button, overlapping tabs, and a lack of a legend for the colorful icons on the map

.


Recommendation 2: Streamlining Map Filters & Search Functionality for Better Navigation and User Experience


We recommend adding a visible search button next to the search bar to make it clear to users that they need to click the button to initiate the search. Additionally, Prioritize the position of the legend, which displays the meaning of the symbols with textual explanations so that users can understand the symbols before they start their search (Figure H). Integrating the legend into the search function as a filter could also improve search efficiency.
To provide more precise feedback for each search action, we recommend displaying the search results in an informative tab below the search area immediately after users enter a keyword (Figure I)
In cases where no relevant results are found; we suggest using a temporary pop-up window to display a clear message indicating that no search results were found (Figure J). 



Problem Statement 3: Image Server Link affects users’ access to images


We discovered a significant issue that prevented all participants from accessing the link to perform the third task in our user testing.

The Image Server Link, meant to provide users with an additional access point to view the images and explore the database efficiently, is positioned under a significant amount of text on the home page under the credit section. (Figure K).
Recommendation 3: Add an access point to the image database on the Resource page with clear labels and improved hover state
We recommend providing an access point to the image database on the Resource page (See figure M).
Besides, subtitles are suggested to be added underneath the title on the Resource page to give users context about what they can find there. 




Problem Statement 4: Home page cluttered with text, difficulty finding image server, information lacks categorization

Image


50% of participants found the home page to be cluttered with excessive text information. During task 3, it was observed that certain participants navigated back and forth to the homepage in search of the image server. However, it appeared that they did not review the homepage content thoroughly, as they subsequently concluded that there would not be an image server available on the home page. Certain information appears to be obscured in the paragraph which lacks clear categorization, making it difficult to locate and access it easily.

Recommendation 4: Simplify and reorganize text, add navigation options, and include more images

To improve the homepage's usability and effectiveness, we suggest three recommendations.

First, simplify the text on the page, use headings and subheadings to categorize information, and make it more readable. The "About" section now includes only the website's core purpose and objectives, while the remaining content previously under the “About” section has been moved to a dedicated "Support" section (Figure P).

Second, add the basic introduction for each section with links or navigation options that allow users to easily move between sections of the website.

Third, include more images to create a visually appealing design that enhances the project's objectives. We have reorganized the content on the homepage and added an "Explore" section with brief introductions to the website's main pages (Figure Q).



Conclusion

The clients were impressed!



A comprehensive report and presentation as well as Figma prototype link were submitted to the client, highlighting the usability study's key findings and recommendations for improvement. The client appreciated the insights provided and acknowledged the value of the new information obtained in enhancing their website.


“Thank you so much for all of this. And we really enjoyed working with you guys. Thank you for making this available to us because I will certainly be using it as we move forward with the project.”
-----Dawn Marie Hayes
“ love the the presentation. Everything was very clear, absolutely. And I liked the summary of recommendations. You know, that's helpful.” 
-
-----Joseph Hayes


Takeaways

During the recruitment process, we reached out to the board members that may have caused some discomfort and confusion. This is a great lesson in how board relationships are very sensitive and clients can be very reactive about them. I will be more careful in the future when it comes to stakeholder relationships.

If we had more time, the next step would be to conduct usability testing on the website’s mobile version. It is now a rough prototype with some issues that needs amendments before engaging in usability testing. If we can make it more easy to use, it would significantly contribute to the exploration of historical monuments while driving or walking on the island of Sicily. 
©Created by Keyi 2024
︎  Contact me
︎  LinkedIn