There is a lot of buzz around ChatGPT and GPT 3.5, but is it really the next Tesla, or is it the next IBM Watson? We talk with HyperDraft’sTony Thai and Ashley Carlisle about OpenAI’s popular tool and why, lawyers at least, shouldn’t be ready to go all in on this specific technology. While there are great examples of how GPT 3.5 impressively handled things like Bar Exam questions, there are still a lot of unknowns from this resource from a company that started out as Open Source and non-profit, but has released a product that is neither.Listen on mobile platforms: 
Apple Podcasts | 
Spotify
Links:
- Bob Ambrogi’s LawNext Interview of Daniel Martin Katz and Michael Bommarito on GPT 3.5’s Bar Results
Contact Us:
Voicemail: 713-487-7821
Email: geekinreviewpodcast@gmail.com
Music: Jerry David DeCicca
Transcript
Marlene Gebauer 0:08
Welcome to The Geek in Review. The podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.
Greg Lambert 0:15
And I’m Greg Lambert. And we’re going to do something a little bit differently this year, we’ll well first of all, happy new year. This is our first show of the year. So it’s a new year back. So this week, we’re going to try and do a little quasi live recording with a couple of guests who we’ve had on this show before or other shows before.
Marlene Gebauer 0:37
Yeah, we’re gonna talk about the cChatGPT and GPT 3.5 AI tools with some experts in AI, Tony Thai, and Ashley Carlisle from HyperDraft Ai, Tony and Ashley, glad you could join us on The Geek in Review.
Tony Thai 0:50
Thanks for having us. Excited.
Ashley Carlisle 0:52
Glad to be here.
Marlene Gebauer 0:54
It’s gonna be a good conversation. Yeah,
Greg Lambert 0:55
it is gonna be really exciting. If if the pre talk is is any indication.
Marlene Gebauer 1:00
Exactly.
Greg Lambert 1:03
Ashley has already put some some restraints on Tony as so. Well, Tony, you’ve been on the The Geek in Review before and Ashley, I still want to thank you for stepping in being my co host for a few of the last episodes of the superhuman legal division podcast. So we’re excited to have you both on here to talk about AI and the GPT tool as it applies to the legal industry. And we’ll even may even try and try a few things on the platform while we’re talking and do some real world world testing if we get a chance.
Marlene Gebauer 1:39
So Tony, let’s start with you and get your initial thoughts on what tools like ChatGPT mean for the industry as of now?
Tony Thai 1:47
Yeah, I think I think the intent behind ChatGPT being a chat bot the the interface, right using chat as an interface between, you know, knowledge database, and and a user trying to extract data from it is super key. It’s something that the industry has worked on for past decade, at least. So folks in the industry are very excited about it. ChatGPT as as applied to the legal industry, I think that the limits, folks are still trying to figure out right now. It my personal view, which is not the most popular view on the mainstream is that like chap, GPT is a little bit of a false flag for folks. Because it, it’s so convincing in its output, that I think folks will want to try to rely on it. But the reality of it is is non domain specific. So it is not trained on just legal information, strain on a bulk of a bunch of information off the internet. And so it, it can lead to some misleading results and confidence in the system. But I still think we’re years out maybe maybe maybe five, five plus years from
Marlene Gebauer 3:04
house we don’t know where we don’t know what it’s pulling from no idea how
Tony Thai 3:08
to do, right, we kind of do. And so like when I when I describe it to people, it’s like, you know, if you’re comfortable with getting your your legal advice from Chora or just from your first five results from Google?
Marlene Gebauer 3:21
That’s true as the first page. Yeah, right. It’s sort of like that.
Tony Thai 3:25
It’s sort of like that. We don’t know. Exactly. Merlin’s point. We don’t know exactly where it comes from, but we kind of have a decent enough idea. And we know it scrapes social media as well. So like, you know, someone’s ramblings from Facebook might be included in that data somewhere, as well. So I don’t know how much, you know, I would want to rely actually, I do know how much I would not rely on it at all. But it poses some interesting questions to the industry as a whole in and, you know, just overall industry. Well,
Greg Lambert 3:55
one good explanation of of why I think jet Chet GPT is getting so much buzz. Is that, for better or for worse, I guess, depending on if you’re Tony, or I guess if you’re me, is this is really the first AI that I think has taken the world by storm. It doesn’t matter if you’re a fifth grader trying to explain a neutron star, or if you’re a lawyer, and you want to see if it can pass the bar exam. This is a tipping point in our culture that I think that makes everyone understand that AI is really here this time, we’re not going to go through I think another AI winter that’s it’s going to be part of our society going forward. And and and, you know, like it or not, I think it does have some viability and a lot of the aspects of our lives and so Ashley or Tony any, do you agree or disagree that that we’re at that
Tony Thai 4:59
point It, it reminds me of when amaz what is it called? Amazon Echo, Amazon. Alexa, Alexa Alexa flip stuff, right? And a bunch of us geeks, were already talking about it. I think some of us were kind of gatekeeping. That’s fine, whatever. But a lot of us were like, Dude, we’ve had this right. We’ve had this for years. But the general acceptance of it came after folks were like, yeah, we’ve had these tools before, but the interface was wrong, right? We tried to get people to use these advanced consoles to control stuff around their house. Turns out people are lazy, and would much rather just talk to it. But similar to Alexa, like, I would say that there was a lot of fanfare for the first year, there’s like people playing games with it. There’s like apps being made for Alexa, it’s since kind of died off. And then the privacy concerns popped up or like, Wait, is his fee always listening to us? Or is it always listening to us? And so then, you know, reality kind of set in after after a while, I think this is kind of the same trend. Just to piggyback
Ashley Carlisle 6:06
on that, you know, this is a great tool. And like Tony said, I think a lot of the adoption, especially the quick adoption is how simple the interface is. But other companies have tried stuff like this. I think other companies have taken different approaches that they wanted to understand kind of the repercussions of the software and put some limitations on it before unleashing into the wild. And this company chose the opposite approach to just like, oh, well, let’s just put it out there and see what happens. So I think like Tony said, like, for now, it’s fun. But as soon as people start really diving in and thinking of the repercussions, and the pros and cons and potential biases within the data, or where it’s coming from, we’re going to have a whole other set of conversations, which I think is bittersweet for us, because we’re very pro innovation and you know, anything we say here, please don’t get that wrong, like we’re very pro moving everything forward. But it’s only going to be a trajectory forward if we also think about it thoughtfully, and make sure people don’t get tired of it or don’t get burned by it. So that’s something percent to see.
Marlene Gebauer 7:03
I mean, did you sit? Do you think that they had any idea that this would sort of blow up the way it did? Because I mean, I know when, you know, when it when it was first announced? I mean, I think all of us kind of went on and, you know, started playing with it. But you know, there were a lot of delays. There were a lot of lags. And, you know, yeah. And and you know, did they did they really kind of think this through in terms of how they released it.
Tony Thai 7:30
I think the strategy was this kind of where they’re leading to right now, which is, recently I heard that they’re in the middle of a fundraiser at least kicked one off. Yep. So as part of their marketing strategy, definitely the board, or, you know, and definitely the executives thought through this and said, This is a good marketing tool for us going into fundraise getting our valuation nice and frothy, so that we can go and raise a lot of money very quickly. I do not think that they thought this would kind of blow up as quickly as it as it did. I think it was planned, I just don’t think it was expected that they were going to, you know, have this level of fanfare so quickly. It’s also I’m really curious, because it’s probably costing them a ton of money, right? Like I that’s the part that like, really confuses me. And like, there’s r&d, and then there’s, there’s beta and alpha user data. But this this, this is an expensive experiment. Well, they’ve already you guys thought about, like, what the point is, I because I’m starting to, I feel like there’s diminishing returns from them on this side?
Greg Lambert 8:32
Well, I guess it would depend, because I think they’ve already gotten a billion dollars from Microsoft. And Microsoft announced, I think, yesterday, the day before, that, they’re going to put 10 More billion dollars into it, and they’re going to embed it into Microsoft Word and Bing. I’m not sure about being but you know, I can see some value of something, a tool like this in Word as as you are writing to have prompts that could that could help you write in really, one way I’ve heard this talk about is this, this is kinda like, maybe the end, maybe this is not the company that does it, because there’s going to be tons of companies that are going to enter the space. copycats. Yeah, and, and so, I mean, it could be the same kind of tipping point, with the investment in these types of tools that were made, you know, the same kind of investments that were made in the solar power industry or electric cars, that, you know, finally AI has something that as make making everyone realized that it’s viable, and yeah, there’s it there’s probably going to be a lot of money wasted. But I think, you know, at the same point, I think with the money coming in, I think you’re gonna see improvement and in the structure of tools like The US and I have a feeling again, there’s gonna be tons of tools like these.
Tony Thai 10:03
Yeah, similar to solar and electric vehicles, it does feel like these guys have a, a pretty large lead and gap over the competition. Because honestly, like if we’re talking about in terms of energy, like data is the oil and the fuel for this. And so it just seems like they have such a huge lead on it. They are open sourcing a lot are not open sourcing, but opening up a lot of their API so that other folks can build off of their, you know, existing models. But yeah, it’ll be interesting. Bing Bing is kind of, you know, we’re talking about marketing approaches. We all know, successful products launch off of being, you know, if you if you’re a successful company, then you better be on Bing. Yeah, I don’t know, a soul that uses Bing willingly. So it’s, it’s kind of interesting.
Marlene Gebauer 10:55
So I want to talk about open AI for a second. So this started off as an open source, nonprofit, but it’s not open source or nonprofit any longer. The back end of Chad APD. And it’s more powerful relative GPT 3.5. There are now black boxes. So, you know, there, it could be, you know, garbage in garbage out. It could be, you know, bias issues, you know, what, what does it mean, for both the open AI and now millions of users? You know, what are they getting themselves into? And do they? Are they aware of that?
Tony Thai 11:34
Yeah, no, it’s it’s kind of what I was getting at before, which is that these large language models, right, so ChatGPT is based off of an LLM, or built off of an LLM, that they’re so massive that the data being audited going in, and then the outputs coming out. Like they’re just these massive machines that folks can’t really wrangle, there are entire new startups built around. Machine learning transparency, trying to give humans a way to view the models and get a bit of a better understanding of it. But Marlene, to the point that we were talking about before, like, we don’t know exactly what goes in. And like one of the issues we’ve seen kind of pop up in a parallel space. Is this generative AI for for art, artwork, you’re seeing it probably not. But there is a lot of outcry from artists, like my friends who are digital artists are not having the the best reaction to it. Because they’re like, wait, they trained on our data, we get
Marlene Gebauer 12:48
paid stuff we created, and then they make money off of that. Yeah.
Tony Thai 12:51
And so as soon as you as soon as Greg mentioned, like the integration with Microsoft Word and Office products, I’m thinking to myself, you know, even as maybe as a law firm, pulled up. So I’m paying all of my salaried employees to feed data into your system. So you can build a product that competes with me, that doesn’t sound like a good yield or output, which is good for, like companies like ours who are, are trying to be more agnostic around and letting our clients build tools for themselves so that they they’re not held down by one, you know, huge monopoly. But it’s, it’s an interesting question, because it’s like, you know, how much of it are they going to cannibalize of their own of their own user base. So with Chachi Beatty, and the way that open ai, ai is structured, one, like you mentioned, it’s no longer open, and so is a for profit venture, which means that they’re going to be a lot more secretive, around what what goes into the model, what the process is like to manage and data wrangle. And then that whole content moderation and output moderation, like it’s largely a black box, we don’t know, what they’re doing, we do know, people are involved. And we do know that they’ve done quite a bit in terms of training and tuning it before they released it, in order to avoid you know, rated our material basically, that goes that would get output from from the model. But now it’s it’s a black box to most of us, we have a general idea of of what goes in and what comes out, but like, we don’t have any control over it. So as what we were talking about, before we started recording, you know, there are inherent dangers on on relying on that type of data because it’s like, Alright, do I do I truly trust it to make decisions for me or just provide research data? So then it then becomes a question for the industry of like, how are we going to leverage this tool, right? Are we going to use it as a spade or we’re going to try to make it into a hammer or drill or, you know, use it for some other function? And Greg likes to eat man Shouldn’t again before the call like, there are parallels for this tool that make it so that it is a useful tool, but you have to understand that it’s not self self guiding, right? Like you need a user behind it. And that user needs exercise judgment.
Ashley Carlisle 15:16
Tony, you can correct me if I’m wrong, but I think something that is leading to the increase like word of mouth about ChatGPT, and kind of more confusion is that it’s trained to seem very credible, it’s trained to seem very competent. And so people will see this on social media all the time, like someone will make a post being like, I feel this way. And this way, here are the pros and cons about shot GPT. Like, you know, what do you think and people are like, well, I did this one thing. And it sounds right. So like, it’s right. And I think that’s what’s confusing is I want people to have that fervor and excitement about use cases and technology. But that’s something I think, if you don’t have a, you know, the background that maybe Tony does, as a, as an engineer, you might not think about well, that’s part of the kind of the DNA of this, it’s going to seem credible, which is why we have to have these conversations about reality where the data is coming from and disclosures as to that and really have these conversations, or else, it’s really easy to be tricked. And then it’s like a slippery slope of dependency on it, without even really thinking about it. I think that’s adding to kind of the confusion, and also the word of mouth explosion here.
Marlene Gebauer 16:18
Yeah, as I say, because up until now, you didn’t have that credibility, like, you know, you would, you would see stuff on social media. And it’s like, okay, it’s trying to write a story. And it was like, all disjointed and terrible. And everybody just laughed, these are undergrad. And if you do that crazy AI, you know, and but now you look at it, it’s like, wow, that makes sense. And that, that seems right. And even if it’s not, right, I can basically say, okay, take that point and expound upon it. And you’re like, that makes sense. And so then everybody’s getting very excited about it to figure out where they can they can use it. And I mean, in law firms are doing that, too. They’re like, how do we, how do we play with this? And to Tony’s point before, I’m like, Are they cannibalizing themselves in the doc in the stuff that they put out there? In order to do this?
Greg Lambert 17:05
Yeah. And just to tie on to that? Well, and I guess this can lead into my next question, as well. And that is, you know, just as we were, as we were talking, I was also thinking about, you know, the data that it’s that it’s compiling the data that it’s using in and the language models that it’s building upon, scraping stuff off the the internet, it, you know, one of the things if you think back about IBM Watson, which that may be another example of of hype, hype hype, you know, excitement. Yeah. And then all sudden, nobody hears about Watson anymore. No, no one seems to, to know what it’s up to these days.
Marlene Gebauer 17:51
For the Weather Channel, some 30 service. Right.
Greg Lambert 17:56
Yeah, I would imagine the last I heard and this was probably pre pandemic, was that it was going to focus in on reading medical journals and working with the medical industry. And I don’t know that that ever panned out.
Tony Thai 18:10
I think IBM wrapped it well, they’ve been same for for this is that’s a great analogy. I, it’s so far removed that I didn’t even think about it. And but the the funny thing is, is it’s so it’s such in the similar parallel, right, like the answer Jeopardy questions really, really well. And in a convincing manner. I totally forgot, that’s a really good reminder, I’m going to bring that up for people. That’s
Greg Lambert 18:35
what we’ll turn to you and mentioned and talked about, you know, who watches the watchmen with that, and Watson in the self contained information, I’ve heard people joke that, you know, the, the thing that’s going to bring something like this down is when it starts producing images of Mickey Mouse, or using Disney characters, or scripts in it that, you know, once Disney gets involved in, then they’re going to kind of get a peek behind the, the black box, but, you know, up until then, you know, kind of who’s, who’s kind of watching what this group is doing, and what information that they may have in their
Tony Thai 19:18
head. Yeah, but that is the joke. Usually, when Disney or Nintendo get involved from a copyright perspective, like you’re screwed, there’s no it’s
Greg Lambert 19:25
a Mario.
Tony Thai 19:27
Yeah. I don’t know. I think, you know, not the reason why I’m pessimistic but when I want to analyze a new technology or any technology for that matter, you know, I can’t help but that you know, we, I come come from a hacker background. And by the way, the word hacker meant something different when I was growing up, it meant more people who would like like to tweak with systems and mess around with them. But you know, you’re always looking for exploits and This system, you know, from a lot of us would appear to be such an easily influenceable system, once we figured out where their data pipeline was. And even if we learned that, hey, you know, they’re getting manipulated by user input directly, you know, a lot of us can already think of, you know, a few different ways to influence output. So that becomes a major concern from a from a who watches the watchmen perspective, like, ya know, the question is, what is that data pipeline going in, where the where the data sources and who’s moderating, you know, who’s monitoring and moderating the, the fuel that’s going into these models, we don’t know, just from the sheer amount of data that’s going into, it would require huge teams of data scientists and, you know, data analysts to wrangle this data to make sure that it’s building something that’s coherent. So that output that you’re seeing that super clean, that’s, you know, convincing, that’s really, really impressive. And I think that’s where, you know, I’m most impressed by this system is how convincing it sounds, not the quality of the data that’s coming out of it, but just how convincing it sounds, that’s really interesting for me, but I am sure, after a year or two on the market, that, you know, certain regulatory bodies will come into play, and that the government will start to get more involved from a policy standpoint, because it would not be in the best interest of the public to just have this, you know, model out there influencing and you’ve seen students use it to write their essays, right. Even from an education standpoint, it’s it’s, it’s, it has to be limited, and it has to be controlled and regulated. So there’s already talk on Twitter, and on other social media platforms, by politicians on how are you going to regulate this? So it’s, so you know, I don’t have visibility on it. Exactly. And I don’t know exactly what’s going to happen, because I still think that technology is incredibly nascent. So I don’t think it can be deployed anywhere for production, except from potentially correcting grammar. And or giving you, you know, essay ideas for for for school school children
Greg Lambert 22:24
will have I’m just curious Have either of you put it to some serious test and found something that you found was useful in the results?
Tony Thai 22:36
Oh, I’ve heard you say, not useful. Actually, I hadn’t if this sounds really bad, but I’ll mention it, we somehow refer a friend it was like he put a he tried to get it to write a eulogy for him. And I’m like, Dude, that stat. That’s like, I got some good inspiration from them. And that’s, that’s good as tough topic. But I, I’ve played around with it, and gotten it to write some and do some interesting research for me when it comes to know some of the stuff I’m working on, but not stuff that would have beaten me Google searching and getting the input from other people. The limitation I find that I find really funny is if you asked a math question, sometimes it gives you the wrong answer. It always fascinates me. But it should give folks kind of the underlying understanding of like, what’s going on underneath, which is it’s, it’s trying to find consensus for an answer, as opposed to doing calculations. And so that’s, that’s a key thing I always suggest people do is ask it like, like, not relatively complicated math question. But asking a math question that, you know, you’d expect a senior in high school to understand and it’ll get almost there. It won’t, it won’t solve it every time. So it’s, it’s quite interesting. Actually, Have you have you had to do any of our marketing yet?
Ashley Carlisle 23:58
No, I haven’t because I tried to get it to answer just a very simple legal question, because I was always just like, oh, how does this work? And I think the thing I realized, which was just a dull moment for me was I hope that this leads people to be more creative lawyers. You know, like sometimes you see briefs that are just like so witty, and so well written that you’re like, Oh, I wish every brief was like this. And I think what I saw was I asked for a very basic legal issue I got a very bland five paragraph like, you know, first year of law school like probably see be a answer. And so I’m hoping that because that is readily available, people will go for the a plus work, or maybe some of like, the low hanging administrative stuff can be done through technology like ours or ChatGPT or whatever, and people can really kind of be like, you know, Ninja lawyers, lawyers have pin with contracts and briefs and what have you.
Tony Thai 24:52
I’m gonna build on Ashley’s point a little bit that’s, that is one of the reasons why I’m so concerned by the level of hysteria or Manea around it is, there is an inherent value in learning how to do it the slow way before you do it the short way. And what I worry about is everybody’s trying to take the shortcuts and say, You know what, instead of doing research, I’m just going to ask this thing, this, and I’m already seeing it with, you know, people that, that I’ve worked with who, you know, don’t take the extra step, or the extra mile to do their own research to understand the underlying case law or, you know, regulation before they come back and just say, this is the case. Yeah, this, this will never replace humans, and I especially think it is, it will never replace, or, you know, have a place in but it will never place a lawyer or a legal professional in the practice of law. The practice of law is a very human practice, I’ve said this before, many times, like, it cannot replace lawyers, because it unless somehow we’ve given up our own destiny or or, you know, sense of control, we just, we won’t, we want to come to the robot overlords and let them dictate, you know, how rules and laws are interpreted.
Marlene Gebauer 26:17
And, and I find your point fascinating, because for so many years, there’s been such resistance to some technological innovations. point being that, you know, we have to teach sort of the up and coming like, they have to be able to do this themselves, and learn this stuff and sort of learn it the slow way, before they can do it the quick way. And yet, that all seems to be like, you know, out the window, just for this this one thing, and, you know, already sort of discussion about like, oh, how can we started, you know, do first drafts and how can we start research this way? And I just find it amazing that, and I don’t quite fully understand why this particular thing. Just just forced, you know, everybody just decided, okay, well, that’s fine. Yeah, there’s so much resistance. Well, I
Greg Lambert 27:08
think actually nailed that, in that it’s very confident and what it gives you, and so I think
Ashley Carlisle 27:13
so. Yeah, exactly, Greg, sorry to interrupt. But I think lawyers like control. And lawyers do not like feeling dumb. And for the longest time, myself included, when you mentioned math or technology to me, I immediately just tense up, I’m like, I don’t want any part of this conversation, right, I think they did a really good job of spending a lot of time making it sound convincing, making it super easy to look at super easy to use, and how the interface is it makes the person feel in control. Like as you’re putting in the prompt, you can immediately given a good point. And lawyers love control, which is a part of our profession. It’s something that’s a neat twist. It’s why we’re successful. I get it. But I think it’s something that’s not talked about enough is we’re impatient. And we’re control freaks. And we need to be cognizant of that as that we’re assessing technology, because people might make these dumb, short sighted decisions, and then regret them later. Because they just want, oh, this is a solution. We’re gonna do this now. And they’re not thinking of the bigger picture, or maybe what they’re bringing that obstacles to the table. So I don’t know why it took off. I just hope he’s the feeling of control, even though they really don’t have control. They don’t have control. There’s no citation of where it’s coming from. Just guesses.
Tony Thai 28:18
Yeah, that’s, by the way, almost impossible. And the funny thing I’ll call out here, because I’d like it DOM documented. schaunard General Counsel always says it doesn’t countless you say it. So I’m gonna go and say it now. That’s, that’s pretty much impossible. Their entire company is dedicated to that citation side that it won’t work. And what the reality around this is. I’ll bring up a great example. Google has built and worked on these projects for years. And you if you want to talk about who’s got the most data, Google has the most data, they have, basically unlimited data, right? And if they’re not deploying it, there’s there’s probably good reason why they’re not deploying it. And at the end of the day, the way that the architecture works is it would have to basically run a Google search at the end, at the end of all of it than just Google for it like it should just, it should just be Google or or a another search engine like Bing, who’s you know, I’ve heard the biggest up and comer out there.
Unknown Speaker 29:14
All the cool kids are using
Greg Lambert 29:19
Bing now was Yahoo. So
Marlene Gebauer 29:25
my yahoo email now maybe it’ll AOL.
Tony Thai 29:29
Right, AOL, hey, you know what I was I moved recently. And I found I had I had one of those old AOL discs for like, their free 20 hours. I know. I left it somewhere and I wanted to frame it up, because I remember I remember that the good old days
Greg Lambert 29:45
MFF. Well, speaking of lawyers using it and and kind of giving it some some testing. If listeners haven’t listened to Bob Ambrogi he’s Interview with Dan Katz and my called Bommarito, about how they put the GPT 3.5 tool, which is different than the the ChatGPT tool, they put that to the task of of answering bar exam questions. It’s a, it’s a good lesson, I think people should, should go listen to what those two did with the tool. And, and they were saying that, you know, the tool has come a long way over the past few years. And, and, and they’re testing. Again, this might be that confidence level. But they were saying that it was not too far away from getting a passing score in some of the the bar in some states. And then Bommarito made a I thought was a really interesting quote. And talking about where we are in the overall process. And he’s saying, you know, change is constant, but the rate of change is not constant now, and it’s not going to change. So we just need to find a way to live with it and make the world a better place. And, and I guess, you know, and I’ve even heard people saying, I texted Marlene, this before. Earlier today was, I heard someone say that 2023 was, was the year that we will finally reach the singularity. And so but I think going back to Bommarito is quote, you know about making the world a better place. Do you think that we have it in us to take tools like this? Let’s say that they do? bring value to it? Are we going to make the world a better place? Are we just going to screw this up?
Marlene Gebauer 31:43
Why do people always think we have the control over that? Like, why? Why is that? Why does that always come up? It’s like we don’t have control over that. People use it for good and bad. Well, also, they
Ashley Carlisle 31:52
gave us no restrictions, right? It’s like, they decided to leech this virus with it to see what happens. See how long it lives.
Tony Thai 32:03
Here, here’s the interesting thing, I forgot who measured it, but we have at our fingertips, way more opportunity than any other generation before except our rate of innovation has slowed down drastically, just drastically. And so like when you look at it, everyone has in their pockets, a supercomputer by 20 years, like 30 years ago, standards has a supercomputer kit. You know, we use it for things like Tic tock and Instagram and like looking at photos and watching videos like that’s the max that we’re going to do there. No, I don’t think will change the world for really materially the better from it, I think, you know, like with any tool, there is great benefit to it. But there’s always a cost. And to me the cost is education and longer attention spans right? To be able to write an essay requires you sit down, read a book, understand and comprehend it, think about it build a mental model, right? Taking the bar is not getting the answer, right, taking the bar is because the test behind taking the bar is not just getting the answer. It’s understanding the patterns and models around the law so that you can answer something when you don’t know how to win, you don’t have the exact answer. And so, you know, I find it fascinating when people run tests like that, because that’s great. Yes. If I had a database of all the answers in the world regarding this, then yes, I could probably answer a good chunk of it. Right. But when human judgment comes into it, I need to convince a jury are convinced the judge or convince the other party of a certain position. AI is not going to do that, right? It’s just not and once it does, it’s that singularity point before her decades from that decades. I think the closest thing that people are missing is actually neuro link. Neural link is probably the closest thing we’re gonna get to that singularity point. Why? Because that would employ a real live working brain to run compute, and our brains, whether people like it or not a hard the best supercomputers in the world right now in the universe. Right? These things we cannot replicate I was discussing with somebody on Twitter. Somebody was like, why can’t we just use Chappie T to replicate a brain and I’m like, that’s insane, guys, like, there’s just not enough compute. Even if there was an if we utilized all the compute in the world to work the way our brain worked, it would be incredibly slow. It would be incredibly slow, even now, that’s how powerful our brains are. And so I think the the takeaway from this is, I’m hoping it’ll inspire people to go out and try to build stuff to better other industries and realize the mistakes that we’re trying to make, which is to jump from A to C instead of, you know, progressing incrementally. So, alright, I’m off my soapbox. I’m sorry. That’s
Ashley Carlisle 34:58
Greg. Once I was Great. Yeah, that was one thing I’ll say I thought it was really interesting that they used I can’t remember, I think it was the commercial version of ChatGPT to answer the multiple choice bar exam questions, right. So one, I’d love to see how it did on the essay portion. I mean, it might have similar results. I don’t know, I know, it’s a lot easier to run all the multiple choice burst. But to I think this is a discussion that’s been happening in our industry for a long time. So I kind of wonder if those very smart guys who I’m big fans of both of them kind of did this on purpose to hit another issue, which is, is the bar exam outdated? Do we need to change what’s going on, especially with technology evolving? Because I think you know, all of us here are lawyers, we all studied for the bar, we could tell people who have not had that experience, a lot of it is just recall, you know, half the test is recall, you memorize it, you put it on whatever. And the other half takes a little bit more skill, a little bit more analysis, what have you. But the part that they tested it on was the pure memorization and recall part of it, there’s a little bit of strategy, but not that much. So I’m kind of wondering if that’s going to lead us to have another conversation in that direction. And that just happens to include shad GPT, as kind of like what gets it in the news. But maybe the bar exam will be for the better from them running those tests. And then also talking about kind of the greater good, so sorry, to jump here. One of the main reasons why I decided to move from big law and a job that I really liked. And, you know, clients that I really liked to joining Tony. And what he was building is I realized that people were going to try to solve these problems, people were going to see money when it came to lawyers and technology and the fact that we’re 20 years behind, and I didn’t want people to just disrespect our industry and not do it the right way. So when I saw people that were really caring about kind of our profession I wanted to jump in. And so that’s something that I’m kind of scared about, because I think Chad GPT hasn’t gotten to that yet. I think, you know, there, I don’t know the amount of lawyers involved. But I really haven’t seen it hit the tipping point where I’m just distressed. But I see other people try to apply that technology and somewhat interesting ways, including like the $1 million SCOTUS wager, and I’m just sitting there, like, Don’t disrespect or make a mockery of people’s blood, sweat and tears for a marketing ploy, because they worked for Elon Musk. And now you want to, you want to come into the legal industry and make a ton of money. Like you got to realize the amount of hardworking people here that you’re kind of the spitting on. And so that’s a big reason why we do what we do every day is like, you know, some days, it’s not fun, but we really do care about lawyers. And we think that even if they don’t understand technology on day one, we want to make sure they’re part of the process. And we want to make sure that we’re the bridge. So I’m curious to see if more people continue to annoy me by trying to jump in and just make a mockery of some things and not understand like, there’s a lot of smart people in the legal realm. We don’t need to explain things to us, we just need you to like, bring us you know, whatever tech solution will tell you yes or no. That’s it.
Greg Lambert 37:54
What about on the other end, we were talking about the challenge, and I’ll let people Google that. But on the other end, I think one of the one of the big issues and and Tony was was joking before we started that, you know that some some of this may be a solution in search of a problem. But one problem that this industry has, and it’s a significant problem is that we don’t have enough lawyers, legal professionals to serve the entire community that that we are supposed to serve. And so there’s so many people that go unrepresented. There’s so many people that need to seek relief from the courts that don’t have the ability or the knowledge or the pathway to get that relief that that they need. Can a tool like this? Well, let me put it to you two ways. Can a tool like this help plug some of that gap? Or can a tool like this actually exacerbate the problem?
Tony Thai 39:07
I don’t have enough experience in like, public interest or even the like, like what my practice was very much like m&a, like low niche work, not very kind of running the mill type of work that most folks would want to consume. So I preface my my thoughts with that. But if it’s anything like what some of these document preparation companies do for filing for, like, incorporation, I’m not gonna name the company because I know them. I like the guys but they create more work for us. When it gets to the lawyers then then help because I’m always fixing the documents, because people always want to do stuff that’s a little bit different and More times than not, it’ll help plug a plug a hole, but it doesn’t solve the problem. I think for people in in need of actual legal help, I worry that this tool would actually make things worse for folks and put them in a bad position because they think that they, they can they can do it without someone guiding them through the legal process. Because it’s not just the letter of the law. There’s, there’s the human component around it. You know, who do you have to get to Who do you have to call in order to get, you know, something filed or something through? When’s the deadline? The deadline says seven days, but in reality should be you shouldn’t send it by six days, because it was a courier, you know, all that stuff, things are waiting at the mailbox. Those are aspects that a quick Google might solve, or a system might solve. But then again, like, here’s my problem, and this kind of outside in approach, there is alternative solutions, right, which is, why don’t we focus on the infrastructure before we look from the outside? So what does that mean? That means improving websites for courts, that means improving our state websites, websites that cover regulation or laws that affect people and that they’re trying to find information for it? Why don’t we work on that, as opposed to trying to go backwards, and make something disorganized, organized, I would say that that’s probably the cheaper, more efficient solution. And we’re starting to see it like I would say, the California Secretary of State website has had major improvements over the last two, three years. It’s very impressive what’s what’s happened as compared to other websites from other states. So stuff like that I see as a more tangible and quicker solution to delivering actual health for folks as opposed to Candy’s Long Shot ideas, because he’s long shot ideas, or they’re not domain specific enough to be helpful.
Ashley Carlisle 41:55
The only thing I would add is there’s obviously a big access to justice problem in this country, and hopefully, with time that will be discussed, you know, law school will be cheaper, or what have you many things to make that less of a gap. However, you know, there are kind of rules in place now, like people represent themselves pro se or, you know, if you’re giving unofficial legal advice, or their stuff on the internet, it’s the disclosure. And it’s kind of the reminding someone that yes, you’re Googling this, but this is a real problem to you, that could have a lot of implications. And I think that is just something that I want included, as people develop these technologies, and they’re applied to kind of that gap is, it is great, we should keep innovating in that space. But don’t forget the basics of what this person is googling could affect their life. And they might not realize if they do it incorrectly, or if they’re late on something, the repercussions of that, and we should have all the disclosures and all the basic things we’re taught in law school, bar exam, you know, CLE, what have you embedded into everything, and that’s something I don’t want lost within the flash of all these new exciting things. It’s like the basics matter, especially once people’s lives
Tony Thai 43:01
died. Can I give you my long shot? Like, vision for Greg? Dream, though, is I was talking to another client, like, a few weeks ago, and he and I asked him, like, why keep bugging me with your legal questions. Now, I don’t say that. But I do ask them like, Hey, why, why did you go with me? And he’s like, Oh, you’re really good at just like, I like to talk to you. And you kind of like my therapist. And I use that phrase a lot is like, Oh, I’m just a business therapist, right, like people call me I most likely have their answer. But the reason why they call me is because I have their answer and can deliver in a way that makes them feel comfortable. I think everybody deserves that from a legal professional. And so the long shot the dream, the vision is that we get away from the billable hour, fix that problem from the law firm standpoint, so that the law firms can employ their their and deploy their their legal resources in a way that can support the community and help with pro bono services. That’s kind of a more for me a realistic dream and solution rather than then try to, like solve everything with technology. I know it sounds weird coming from a tech company. So
Greg Lambert 44:12
I laugh because my Facebook reminder showed me that 14 years ago today, I posted four articles about killing the billable hour. And it’s just as strong now if not stronger than it was 14 years ago. But one of the things that I’ve kind of heard and it could be pie in the sky, type of hope, hopefulness on this is that tools like this and even tools like like yours, Tony would give us the ability to take back some of our time. And if we can take back some of our time using automation, automating things that should be automated sets. No more argument, then we can make that transition away from charging for the billable hour and and then have that ability to use whatever excess time we have now, to do what you’re saying to build those relationships to be that counsel, to be that trusted counsel to people, and not feel like they’ve got the anvil of the billable hour hanging their head, in order to achieve that very valuable goal that we want to do. So, you know, I think we’re not there yet. I think we’re probably going to fall down before and look really bad before we can figure out ways to actually use tools to help us take back our time and a valuable way such as in order to realign our values away from from the billable hour. So what do you think? I mean, tools tools like yours, or is that kind of the goal that that you have on that? And how should law firms be looking at technology as it comes out? And is trustworthy? And using the tools and effective way?
Tony Thai 46:24
Yeah, so two fronts, one, I think, I’ve done, I’ve talked to a lot of law firms in the abstract, right, talk to them as a as an entity. And from our, from our experience, it’s not worked, right. And, you know, when we took a step back and did the post mortem on it, and we realize, the reason why it doesn’t work is when you say, firm, it’s this, like a amorphous, like very large ship that you think is all rowing in one direction. And it’s not, right, it’s a lot of sub cells of people were just trying to make their day to day work. And they’re all doing different things, all at different times on their own way. And so when we stopped looking at the firms as like, large ships, and we started thinking about them as fleets, right, and so you know, you’ve got different battalions in different spots, and they have a different way that they like to operate. We’ve gotten more surgical with it. And so when I talk to the firm’s, I tell them hey, like, don’t tell me what you think you did, the firm needs. Tell me what you think one group needs one group at a time, right? And we solve it one group at a time. And you know, who’s who’s the squeakiest wheel, because though, they’re most likely going to adopt a tech, if they’re complaining to you the most, and they’re complaining to you in the most of your staff and personnel probably annoyed by it. And so maybe we can solve that problem. And so we go in there first, I see you nodding your head. Yeah, that’s it’s it’s most likely those groups. So my recommendation is finding the incremental improvements, and building on those. And Greg, the reason why, like just Greg and Marlene right, the reason why like, I’m jaded by some of these things is I’ve seen it in the engineering field, where we are today, in 2023, is where we were 15 years ago, I kid you not like from a technological perspective, we haven’t improved that much from a programmer and engineers perspective. we’ve reinvented the wheel like two or three times, and then we’ve gotten back to this point. And then everyone’s just like, oh, it’s all cool and new. I’m like, Dude, this was here. 15 years ago, it’s remarkable that people have such short term memories. And to your point, like we’ve been talking about building innovation for three decades now. Right? It’s no longer innovation. It’s now like this is this is just, it’s it’s a broken record at this point. I don’t think that there is a clear cut answer to the billable hour. I think that from a customer perspective, like consumers of outside counsel, and from, you know, the law firm, billable hour, we find it all very funny that folks bill by the hour, because we actually just, you know, I ran an in house team, like, we paid what we thought was fair, and what we agreed to with folks, I never really thought about the billable hour, unless it was a second tier law firm. That was promising me value on the cost side versus quality. But at the top tier firms like Mayor Brown or Jackson, like those relationships that you built with clients as their business therapists and consultants like that, that doesn’t get measured by by, they gets measured in part by the billable hour, but the real performance is your output and what they feel they got in terms of value out of you. And so I don’t think that that’s a technological solution that when that one’s just a realization on the business side, and we’re seeing by the way, just to be clear, like our clients, our law firms that our clients are starting to move in that direction. We have plenty of clients that are billing flat rate. For for for matters and Using our tech to help deliver that value chain and increase the quality of life for their for their people. And so there’s like, there’s like 50 different factors that come into play. But, and I didn’t answer your question directly, but because I rambled for too long. But yeah, I would say when we looked at firms like, we stopped thinking about them as these ginormous entities, and we started looking more at the individual. And tools like these, like Chechi Beatty, that can empower the individual definitely will help. It’ll just help inspire them to want to improve their process. And it might not be Chechi Beatty, that’s the thing that excites me the most about the spec is it gets people inspired. And they’re like, dude, this can actually help. And then they realize a limitation that like, you know what, maybe I can do better about setting up a system. And being more organized about how I reuse these forms with these data. Now we’re seeing clients come to us with a whole action plan. That’s, by the way, brings me so much joy. It also removes a lot of headache when clients come to us and they said, we’ve seen this tech, this tech, this tech, this is what we want you to do here, here and here. Can you do this, this and this? That’s beautiful. We’ll flick three years ago, folks were coming in and be like, I have no idea what we’re doing. Can you just automate all of it? Which is just an, you know, unrealistic approach?
Greg Lambert 51:19
Well, I think our whole audience was shocked when you said, big law firms are not smooth sailing vessels. I understand.
Tony Thai 51:29
You know what, there’s, there are many folks in law firms that think that way. And it surprised me because once they get to partnership, they’re like, oh, what this this is what, this isn’t what I signed up for you guys Robbins,
Greg Lambert 51:42
where did this coming from? No, no. This
Marlene Gebauer 51:45
is just a microcosm. And then all of a sudden, they see the bigger picture. It’s like, oh, that’s a shocker,
Tony Thai 51:52
though. Exactly. Yeah, I mean, my final point with it is like, despite my negativity, I’m very excited by the Tech because like what I said before, it inspires people to want to improve. And that’s better marketing lane and a use of time and resources for everyone. Right? Like if he if he can inspire people to get better. I’m cool with that or and want to build better solutions that makes brings me a lot of joy. Like what, what I worry about and what I want to warn people of like, if it sounds too good to be true, it probably is. And so like, Guys, come on, like, get excited, but at the same time, like, just temper your expectations around what it can can and can’t do is just trying to be the adult in the room.
Marlene Gebauer 52:39
Gonna say just like your parents taught us and just like our parents taught us. Yeah,
Tony Thai 52:42
yeah, I was terrible at that just.
Greg Lambert 52:47
All right, well, Tony Thai and Ashley Carlisle from HyperDraft. Ai. I want to thank both of you for coming in and having this roundtable discussion. This is this has been great. I’m probably still drinking a little bit of the Kool Aid, but I’m much more self aware now.
Tony Thai 53:03
Yes. Thanks for having us. Thank you so much.
Unknown Speaker 53:06
Thank you. Super fun.
Marlene Gebauer 53:07
So thanks to everyone for taking the time to listen to The Geek in Review podcast. If you enjoyed the show, share it with a colleague. We’d love to hear from you. So reach out to us on social media. I can be found at gay Bauer am on Twitter,
Greg Lambert 53:19
And I can be reached @glambert on Twitter. Tony, where can they find you?
Tony Thai 53:24
Actually what we’re working, they find us
Ashley Carlisle 53:26
follow us on HyperDeck you can follow us at HyperDraft Inc. on all social media. We especially love LinkedIn. You can find Tony at Tony Thai. He’s also linked on our Twitter account at HyperDraft Inc. And you can check out our website hyperdraft.ai.
Marlene Gebauer 53:42
If you have any comments about the podcast, you can leave us a voicemail on our geek and review Hotline at 713-487-7821. And as always, the music you hear is from Jerry David DeCicca Thank you, Jerry.
Greg Lambert 53:55
Thanks, Jerry. All right, Marlene, Tony, Ashley, we’ll see you guys later.
Marlene Gebauer 53:59
Okay, bye bye. Back dibbles back and back
