Hoppa till innehållet
Dialogica Foto: Peter Parnes
Dialogica med 5 användare. Foto: Peter Parnes Visa originalbild , Öppnas i nytt fönster/flik

Dialogica - en interaktiv samtalsapplikation i en dataspelsmiljö

Publicerad: 28 oktober 2020

Dialogica är en applikation som utvecklas för att hjälpa personer med talsvårigheter att uttrycka sig i djupgående samtal via grafiska avatarer.

I Dialogica kan användarna chatta, få meddelanden upplästa (text-till-tal), få meddelanden översatta till sitt eget språk, röra på "sig" via animeringar, välja vilken avatar som ska representerade "mig", byta värld där samtalet pågår med mera.

Dialogica utvecklas med grund i samtida forskning inom dataspelsteknik och inom filosofiska samtal. Den pedagogiska funktionaliteten baseras sig på teorier och metoder om effektivt och högkvalitativt samtalsdeltagande och samtalsledande inom den pedagogiska metod som ofta kallas ”filosofiska samtal”, och från argumentationsteori.

Applikationen utvecklas i dataspelsmotorn Unity och genom att denna används kan applikationen göras mer lik ett dataspel även om just spelmekanismer inte används (just nu). 

Applikationen är utvecklad primärt för iPads (nyare sådana pga krävande grafik) och kan även användas på nyare telefoner. 

Dialogica kommer att provas med användare med Afasi under vintern 20/21. 

Se även arbetet som pågår med applikationen WalkAbout som är en applikation som utvecklas parallelt med Dialogica där användaren istället kan röra sig i en 3D-miljö.  

Publikation "Game technologies to assist learning of communication skills in dialogic settings for persons with aphasia" från 210212 om Dialogica i International Journal: Emerging Technologies in Learning, ISSN 1868-8799, E-ISSN 1863-0383, Vol. 16, nr 03, s. 190-205. 

Här följer en lista på ändringar per version. Denna lista kommer att uppdateras varje gång en ny version släpps. 

0.12, 200519 

  • Update UI with settings panel
  • Moved change name into settings panel
  • Added TextToSpeech including settings
    • User can select if to speak messages either local, remote or both
    • Messages are buffered meaning that they will play after each other
      Users can select from downloaded voices. New voices are downloaded in settings in iOS.
      ​Swedish: Hjälpmedel -> Talat innehåll -> Röster -> Svenska
      English: Accessibility -> Spoken Content -> Voices -> Swedish
      Most voice are available in both bad and enhanced versions. Enhanced is better.
  • Default ChatMessage timeout is now 30s.
  • Settings now show it we are the server
    • If we are the server and quit the session will be destroyed for everybody. To be fixed later but this is a warning for now.
  • Updated frame rate to 60 frames per second to give smoother animations on mobile devices.
  • Last selected avatar is saved in preferences and restored on next session.

0.13, 200604

Graphics:

  • Changed to new rendering engine (URP) which gives better graphics and crisper colors.
  • Moved the main light (the sun) to better match environments.
  • Updated worlds to latest versions (no major visual changes)
  • New horizon in the city scene.

UI:

  • Complete modal window handling (only one top level window visible at once)
  • Added close button to chat window
  • Using same world when we restart if we are the server (or else we listen to the server)
  • Send button in chat window is now only enabled when it should be.

Content:

  • Changed order of animation buttons and a few of the messages.
  • Not repeating spoken messages when "spamming" the anim buttons. Only one message from the anim buttons can be in the queue. Chat-messages are allowed to be repeated.
  • Experimenting with changing words that are to be spoken to make them sound better in Swedish.
    E.g. "förargument" to "för, argument"
    "förexempel" to "för, exempel"
    Still not satisfied with the results though but will have to do for now.

Other:

  • Moved to Unity 2019.3.15

0.14, 201002

UI:

  • Highlighting current selected button at level 2 when composing message.
  • Added next/prev buttons for Avatar and World
  • Showing window on startup when we are searching for server.
  • Removing UI-buttons on startup.
  • Showing message when we lost connection to the server and an OK-button to quit.
  • In the future this should restart the server so we can continue.
  • Added version and build time info to settings window to easier distinguish between different versions.

Content:

  • We can change name again. Bugfix.
  • Updated text for chat message for button 9 (Vi behöver...)
  • X-chat message now removes earlier chats. Clear.
  • Names can now contain numbers. Bugfix.

Other:

  • Update to newer network code (Mirror 19.0.3)
  • Update to Unity 2020.1.6.

0.15, 201015

UI:

  • Closing all other windows when we show "lost connection to server"-window.

Content:

  • Added realtime translation via Google Translate incl. UI for selecting language and if we should translate or not.
  • Added support for 5 participants and redid start position handling.
  • Saving AvatarIndex that is randomized on fresh start.
  • Changed default name to Avatar instead of Player (plus number).

Other:

  • Update to Unity 2020.1.7f1

0.16, 201028

Highlights:

  • Own messages with lock (double tap) and delete (long press)
  • UI changes: green->black/white, icons for avatars and worlds
  • Spoken content: speed slider added, sorting fixed
  • Swap of avatars. Double tap the name tag on the first and then double tap the second.
  • Camera allows for 5 participants (changes dynamically)
  • Various fixes for real-time translation.

More detailed changelog

UI:

  • Camera zooming out when we are more than 3 participants (max 5 for now)
  • Added icons to buttons for changing world and avatar.
  • Added delete of own buttons by holding the button pressed.
  • Languages are now sorted in the language dropdown.
  • Changed all colors in UI to black except white on the startup screen.
  • Added buttonLocking for own chat messages via Double Tap.

Functionality:

  • Handling when translation is not available. Just use the original language string.
  • Added own chat messages. Input field works with speech as well via iOS.
  • Added persistent cache of translations. (Translations are saved to disk)
  • Added speed slider for spoken content.
  • Added swap of Avatars.

Other:

  • Typos in language names fixed
  • Added API protection for translate API.
  • Updated to Unity 2020.1.10

0.17, 201116

Highlights: 

  • Added TreeView, open via T in upper left corner. 

Tree: 

  • Initial test release including many new features including shared tree, loading/saving tree, zoom, change link color, adding and removing nodes etc.  

UI: 

  • Moved speak messages toggles in settings around. Added back the FPS-toggle to test performance on different devices. 
  • Color changes to name plate. Others name is now black and not bold and chat text is black. Removed colors for own name on name plate and setting darker background for own name plate.
  • Fix for bug where locked chat button stayed locked after deleted by long press.

Functionality: 

  • Added more startpositions (now 7) and redid camera handling when switching between 3, 5 and 7 participants.
  • Added "Jag vill få en fråga" in chat-panel-1.

Other: 

  • Update to Mirror 26.2.2. (new network code)
  • Updated to 2020.1.11f1

0.18, 201130

Highlights:

  • New room setup that should work through firewalls.
  • Many changes to the Tree.

Network:

  • Network traffic will go through either the relay server or directly if the clients can connect directly to each other.
  • Added DarkRift-relay and setup new relay-server.
  • Changed to newer KcpNetwork-protocol for communication between clients.

UI:

  • Added button in settings-window for leaving room which will push the user back to the "select room"-screen.

Functionality:

  • See above.

Other:

  • Updated to Unity 2020.1.15f.

Following are all the changes to the Tree:
I/O:

  • I/O port positions switched, O is on the top of the node, I is on the bottom.
  • Input ports now support multiple connections.
  • You can now drag connections from I to O or O to I.
  • I/O ports are now light pink, removed options to change its colors.
  • Increased the size of I/O ports to easier make connections using touch.

Modal windows:

  • Only one modal window open at a time in treeView.

Zoom:

  • Now starts slightly more zoomed in
  • No longer save the zoom-level between sessions, perhaps better this way.

Node:

  • Now defaults to switching text instead of colors when using the "Ändra" button.
  • Spawning nodes via the + button on another node now spawns further away from it's origin node.

GraphData:

  • Removed I/O colors from GraphData, all saved trees will be removed first time you start with this new version of graphs.

Bugfixes:

  • You can no longer attach connection between I/O of the same node...
  • Unsubscribe action events upon leaving a server/room (tree events works properly when re-joining a room).

Technical details:

  • xNode is no longer a package but imported under assets. This because scripts has to be modified to suit the needs of the project. (I/O rework).

0.19, 201211

Highlights:

  • New create room UI.
  • Updates to network handling for rooms (join/leave etc).
  • Numerous changes to the Tree. See below. 

Network:

  • Added threading for reconnecting to Relay-server for server list.
  • Bug fix for connecting to the right server when joining a second room. Earlier we could end up the previous room by mistake.

UI:

  • New create room UI.
    • We save the last room name.
    • The "right" info is shown in the popups.
  • The settings window now show the name of the room we are in.
  • The X-chat-button now clears the speach-window.

Other:

  • Update to 2020.1.17f1

Following are all the changes to the Tree:

Zoom:

  • If holding either zoom button for 1 second the zoom will reset to initial zoom level.

GraphData:

  • Changed data structure, all old trees will be removed on first bootup.

UI changes to nodes/links:

  • Created 2D sprites to be used as different shape of connections between nodes. (Sawtoothed, waves, balls, dashed)
  • Removed the choice of individual colors, instead choosing a type such as "förargument", "motargument", etc, will color the node and the links as well as the links shape from its output.
  • This choice can be made for individual links going out from a node by clicking the link.

Node spawn:

  • Spawning a node via "+" on another node now has 5 different spawn locations to minimize risk of overlapping.

Minor fix

  • The load panel now close when loading a tree.

0.20, 201221

  • New UI for choosing world and avatar (WheelUI)
  • Better camera handling without fisheye effect and also scales down to fewer participants when suitable. 
  • Bug fix: Not running at top speed in the select room view.  

0.21, 210118

Several updates to the tree including: 

  • History of actions in 4 steps (rewind and forward) 
  • There's now an autosave file accessible from "Ladda", the tree is autosaved every time there's a tree modification that's not a load/save action. (A new node, a new connection, change of type, text, etc...) 
  • The tree is now a lot more responsive / fluid, overall the runtime graph should be more robust.
  • Nodes are now destroyed properly when treeView is hidden, this has removed all multiplayer bugs
  • It is now possible to drag connections from an output port to connections (Excluding connections originating from the same output port)
  • There's no limit to the depth of connections between nodes and connections. (i.e if you drag a connection from a node to a connection, you can drag a connection between another node and either previous connections.)
  • Note that any old saved trees will be removed upon first bootup.

Other:

  • Added round corners to ImageButtons to match the normal buttons
  • Updated to Unity 2020.2.1f1

0.22, 210120 

Tree

  • Fix for broken tree handling. 

UI

  • Now you can select avatars by clicking on them and the wheel UI will scroll to that avatar. 

0.23, 210128

Overall:

  • Voice chat added. For each avatar incoming audio will highlight the avatars head with a blue dot and a light. Works and can be tested! Note: currently now indicator of outgoing audio (will come in next build). 
  • Added backgrounds to the Avatar and World choosers. 
  • Added graphical icons the Tree, Chat and Voice. If you have suggestions for other icons please let us know. Now they can easily be swapped. 

Tree: 

  • Nodes now have a chat button to open the chat, if "jag vill ge/få... motargument/förargument/fråga" is sent through the node chat button, a new linked node with the type motargument/förargument/fråga will spawn.
  • Added notifier in the main UI to show something has happened in the tree. 
  • You can now open the chat via tree node.
  • Improved spawning of new nodes to prevent "perfect overlap" of nodes, currently checks the position of all its children to make sure not to overlap one of its children. (This can easily be improved to check against all nodes and check an area instead of a point, feedback/discussion required).

0.24, 210212

Overall: 

  • New UI icons for tree and settings.
  • Added anti-aliasing and ambient occlusion for nicer graphics.
  • Filtering away WalkAbout sessions in the launcher. I.e. we only see Dialogica sessions.
  • Worlds are now dynamically loaded which means less memory usage but gives a very slight delay when switching worlds.

Tree: 

  • Adjusted touch area sensitivity when dragging connections from nodes and to nodes.
  • When you select to edit title or body text of a node it will now automatically zoom-in to focus on the node.
  • When you finish editing title or body text of a node it will now automatically zoom-out to the view before editing.
  • The "Nytt träd" button will create a new tree only if the current tree is not empty.
  • Removed the confirm/decline panel for creating a new tree.

Technical details:

  • Updated to version 7.2.0 of Dissonance (VoIP-library).
  • The zoom functionality in the tree should make it easy to create further zooming functionality (100% view of tree, pinch to zoom...)
  • Restructuring of various packages to make it easier to reuse in WalkAbout.

0.25, 210225

Overall:

  • Fixed bug where settings wouldn't close when opening Avatar and World choosers. 
  • The network code has been updated to new protocol and new relay server. 

Tree:

  • * UI button functionality
    • "Ny nod" button now checks for perfect overlap.
    • "Nytt träd" button now creates a new tree with a new node and zoom-focus onto the node.
  • Chat updates
    •  "Jag vill ge...", "Jag vill få...", each option will now spawn a node of matching type.
    •  Chat via treeView will close upon sending a message, chat via mainScene will not close upon sending a message.
  • Auto-zoom
    •  Improvements to auto-zoom on nodes, more dynamic scaling to fit more screen resolutions & more minor fixes.
    •   100% overview button now available between the two zoom buttons, note that it will only zoom out to fit, it will not zoom in.
  • Minor fixes
    • Moved the tree notification counter to fit the new tree image.
    • Changed the "None" node type color from Cyan to Gray.

Technical details:

  • New network code (KCP -> SimpleWebProtocol for now) 
  • Showing build info in Lobby-scene.
  • Unity 2020.2.6 
  • Reworked file storage for graphData.
  • - Moved the xNode package to the internal 'packages' directory.

0.26, 210313 

Overall: 

  • Showing dialogue window when we lost contact with the server.
  • Shows a message when loading servers initially in the lobby.
  • Closing microphone when not sending. 
  • Volume meter on the microphone showing if we are actually sending or not. 
  • Automatic echo cancellation added (works good on mobile but needs more trimming on desktop). 
  • LTU-50 shown as background when launching the app together with Unity logo.  

Tree: 

  • 100% zoom on tree, via button, on load and on initial open of the tree. 
  • Move of subtree with 2 fingers on a node (1 finger still moves just the one node) 
  • Scaling of links in tree is now correct with different resolutions. 
  • Updates to auto zoom of tree. Should work fine now. 

Other: 

  • Unity 2020.2.7f1

0.27, 210423

Overall

  • Step 1 of profiles system
  • Major performance increase due to graphics changes. 

Local profile system

  • Teacher & student profiles
  • Create & delete profiles

Permission system

  • Student profiles receive basic permissions upon creation
  • Teachers can toggle students permissions to enable/disable functionality
  • UI handler greys out disabled functionality

Tree

  • Pinch to zoom: You can now use pinch to zoom in/out on the tree (it will zoom in/out of the middle of the screen)
  • 100% treeOverview: It will now also zoom in to fit the tree
  • touchKeyboard Overlap: When selecting an inputfield it will now check if the inputfield is covered/behind the iOS keyboard and adjust accordingly.

Known issues

  • Anyone can create/delete/login any type of profile.
  • No state checking (i.e, if a teacher disable a students text chat, if the text chat is already open, the student can still use it)
  • The permission system don't handle when two students have the exact same name.
  • If two teachers are toggling the same student's permission, their UI doesn't update when the other teacher toggle.
  • Toggling VoiceChat permissions off/on breaks audio sending. 

Minor UI issues.

  • Zoom in to fit tree doesn't always center properly if extremely zoomed out.

Technical Updates 

  • Unity 2021.1.4 
  • Newer network code 
  • Major changes to graphics to increase performance including static mesh batching reduced 80-95%, reduced number of light calculations, reduced antialiasing to 2X and turned off HDR.  

0.28, 210520

UI

  • Updated profile system UI (Bigger buttons, better navigation...)
  • Creating or login teacher profile now requires code "XX" (temporary code, contact team for code)

Profile & Auth system

  • Settings are now also attached to profiles.
  • Now checks against playerID on server instead of player name.
  • Disabled functionality is now removed instead of greyed out (for students)
  • Can now set a default set of permissions when creating a room.
  • Can toggle a set of permissions for a student in addition to one seperate permission.

Bugfixes

  • Fixed textToSpeech voice selection (always defaulted)
  • Additional UI syncing between teachers when changing student permissions.
  • "B" button now properly hides when choosing world/avatar.
  • "Välj värld/avatar" window now doesn't disappear after choosing world/avatar the first time.
  • Various crash fixes for the tree (All that has been reported)

Known issues

  • No state checker for disabled functionality.
  • When toggling one permission, the shown "set" of permissions remains the same.
  • "B" button should be changed to an image.

Technical issues

  • Unity Editor must be re-compiled manually once on boot to be able to load trees.

0.29, 210924

Highlights:

  • Crossfading animations instead of playing them directly. Looks much better.
  • Flashlight for the tree including colors on chat bubble.
  • "all permissions" the default room option.
  • UI updates of the tree nodes (scrolling, color of ports etc.).
  • Hiding our own room in the room list so it doesn't show up right after we have left (killed server) and try to rejoin the room.
  • Network changes, faster join, better handling on errors and on leave.

Player Canvas (Chat bubble)

  • Added frame & knob with a randomized color for each user upon joining a room.

Tree Highlight Mode

  • User can now toggle between Drag & Highlight mode.
  • In Highlight mode the users input will spawn a "flashlight" effect for everyone to see.
  • The "flashlight" effect is color matched with player canvas and displays the player name.

Permissions & Profiles

  • Made "all permissions" the default room option.
  • Added highlight mode as a permission for teachers to toggle.
  • State checking (i.e, if you disable a student's voice, they stop broadcasting their voice)
  • There's now an "Alla Studenter" which allows you to toggle permission set or individual permissions for all students at once.
  • UI fixes and syncing between teacher clients.

UI

  • Player Canvas (Chat bubble), added frame & knob with a randomized color for each user upon joining a room.
  • treeNodes body text is now scrollable.
  • treeNodes ports are now white.
  • Chat "Annat" button texts has been rearranged.
  • Permissions button is now an image of a key.
  • Crossfading animations instead of playing them directly. Looks much better.

Technical Details

  • Refactored Profiles & Permission systems.
  • Added editor scripts.
  • Editor bug to load tree's before force compiling scripts is gone.
  • Cleaned up text & UI handling (overflows, etc)
  • Minor bugfixes and clean up.
  • Updated network code.

Kontakt

Peter Parnes

Peter Parnes, Professor

Telefon: 0920-491033
Organisation: Distribuerade datorsystem, Datavetenskap, Institutionen för system- och rymdteknik
Ylva Backman

Ylva Backman, Universitetslektor

Telefon: 0920-491289
Organisation: Pedagogik, Pedagogik, språk och ämnesdidaktik, Institutionen för hälsa, lärande och teknik
Viktor Gardelli

Viktor Gardelli, Universitetslektor

Telefon: 0920-491550
Organisation: Pedagogik, Pedagogik, språk och ämnesdidaktik, Institutionen för hälsa, lärande och teknik

Taggar