@ychen , I see. I was expecting your findings were a part of the phd program. Take your time with publications then, since it is common while at Phd. It would be great to have a paper during the masters and all the best with it!
@nicolay-r my master was in a slightly more conventional project, a randomized controlled trial. Definitely hoping for this project to be part of my PhD, but we’ll see how the proposal goes 😂
@ychen Good luck with your studies and pleased for affecting on your advances in it. Are you on google scholar or github with personal advances in this domain?
Nothing published yet. Started this line of work around 2 years ago during my masters. There’s a lot of challenges and I’m trying to reach out to people. Would love to connect with you though! Just followed you on bsky:)
@nicolay-r thanks for sharing those articles! Pretty relevant so I'll definitely check it out. Yes, my focus is more on the clinical side, and clinicians are not happy about this kind of broad characterizations. Not sure what's causing it, but this behavior seems to have seeped across models from different companies, so a way to analyze more would be very helpful.
Thank you @ychen for sharing this! I was curious, because the word freq analysis you're attempted to do is very aligned with lexicons construction and frames in the domain of sentiment analysis. In particular, this could be enhanced up to analysis on a specific set of words, usually dubbed as frames. So and unlike just words, frames goes further with sentiment of subject towards objects.
FYI. We cover the similar for news and domain specific (Russian language) here: https://github.com/nicolay-r/RuSentiFrames
@nicolay-r Thanks for sharing as well! I’m actually largely working on post training language models for mental health purposes. To make sure I understood you correctly frames are basically describing how a sentiment is related to entities in a sentence—is this a roughly correct understanding? I checked out your work on targeted sentiment analysis and I think it might be interesting for my work as well!
Hey, I just commented on PR #35466 and would love to contribute!
Heyy I think you commented on the wrong post!
Thanks! Any publicly available resources of such a synthetic texts that would lead to your observations?
@nicolay-r If you are asking for prompts to elicit these responses, it's just based on my impression from testing it with different prompts over the past year for now.
Here's a few to try that should pretty reliably trigger "tough" or "rough":
I'm planning to build a synthetic dataset related to this, but if you want to do some quick word freq analysis, try running sentiment filter over chill-bots/empathetic-dialogues-cleaned :)
