1
00:00:00,480 --> 00:00:03,480
foreign

2
00:00:09,800 --> 00:00:14,460
I would like to introduce your speaker

3
00:00:11,940 --> 00:00:17,100
this morning uh Tom Eastman

4
00:00:14,460 --> 00:00:19,619
Tom works for the Kraken sponsor that we

5
00:00:17,100 --> 00:00:22,080
have and he'll be he has attended every

6
00:00:19,619 --> 00:00:23,939
Pike on Australia we've had so far

7
00:00:22,080 --> 00:00:25,980
I think that's 10 years worth which is

8
00:00:23,939 --> 00:00:28,080
great

9
00:00:25,980 --> 00:00:30,300
um Tom loves to talk about many many

10
00:00:28,080 --> 00:00:31,800
things his great person to have

11
00:00:30,300 --> 00:00:33,300
conversations with so if you do want to

12
00:00:31,800 --> 00:00:35,640
talk more about this talk afterwards

13
00:00:33,300 --> 00:00:38,219
find him at the Kraken booth at

14
00:00:35,640 --> 00:00:39,180
lunchtime and he will happily talk your

15
00:00:38,219 --> 00:00:41,820
ear off

16
00:00:39,180 --> 00:00:44,520
I'll let you go if you need to

17
00:00:41,820 --> 00:00:46,100
um please provide a warm welcoming

18
00:00:44,520 --> 00:00:49,459
Applause for Tom

19
00:00:46,100 --> 00:00:49,459
[Applause]

20
00:00:49,860 --> 00:00:55,620
hey I'm Tom um who has heard me talk

21
00:00:52,860 --> 00:00:57,360
about secateur before in this room

22
00:00:55,620 --> 00:00:59,399
quite a few of you cool cool because

23
00:00:57,360 --> 00:01:01,920
it's this is the third talk that I've

24
00:00:59,399 --> 00:01:03,960
given on this what was just like a silly

25
00:01:01,920 --> 00:01:05,519
little Django side project

26
00:01:03,960 --> 00:01:06,540
um that kind of grew a little bit out of

27
00:01:05,519 --> 00:01:09,000
control

28
00:01:06,540 --> 00:01:10,560
and so there's a lot of detail that I

29
00:01:09,000 --> 00:01:11,880
might skip over and I'm more than happy

30
00:01:10,560 --> 00:01:13,439
to talk about that stuff or you can

31
00:01:11,880 --> 00:01:14,700
check out my previous two talks on the

32
00:01:13,439 --> 00:01:17,520
subject

33
00:01:14,700 --> 00:01:18,960
um just a bit of a Content warning this

34
00:01:17,520 --> 00:01:20,400
is a talk about a tool to protect people

35
00:01:18,960 --> 00:01:22,320
from social media harassment so I'm

36
00:01:20,400 --> 00:01:24,479
probably going to be mentioning

37
00:01:22,320 --> 00:01:25,979
some of the things some of the sorts of

38
00:01:24,479 --> 00:01:26,759
harassment that kind of happen on

39
00:01:25,979 --> 00:01:28,500
Twitter

40
00:01:26,759 --> 00:01:29,460
um I'm not going to dwell on any of it

41
00:01:28,500 --> 00:01:31,380
but that means that there might be

42
00:01:29,460 --> 00:01:32,939
mentions of transphobia and homophobia

43
00:01:31,380 --> 00:01:34,560
and stuff like that just because that's

44
00:01:32,939 --> 00:01:36,060
the kind of awful [ __ ] that happens on

45
00:01:34,560 --> 00:01:36,659
Twitter

46
00:01:36,060 --> 00:01:38,400
um

47
00:01:36,659 --> 00:01:39,659
the Project's dead now and we're going

48
00:01:38,400 --> 00:01:41,280
to go into detail we're going to go into

49
00:01:39,659 --> 00:01:44,040
a little bit of detail on that so this

50
00:01:41,280 --> 00:01:45,479
is kind of just a recap of why I built

51
00:01:44,040 --> 00:01:49,079
it what I learned along the way it's

52
00:01:45,479 --> 00:01:51,540
kind of a grab bag of of anecdotes and

53
00:01:49,079 --> 00:01:53,520
things that happened maybe some of this

54
00:01:51,540 --> 00:01:55,380
will be useful lessons

55
00:01:53,520 --> 00:01:56,640
um maybe it won't be some of it will

56
00:01:55,380 --> 00:01:58,220
hopefully be a little bit entertaining

57
00:01:56,640 --> 00:02:00,960
some of it will be a little bit surreal

58
00:01:58,220 --> 00:02:03,540
I'm going to try

59
00:02:00,960 --> 00:02:06,780
to play a tiny bit of a question that I

60
00:02:03,540 --> 00:02:10,319
received at 2019 when I first launched

61
00:02:06,780 --> 00:02:12,780
this uh tool

62
00:02:10,319 --> 00:02:14,760
you there we go have you done any threat

63
00:02:12,780 --> 00:02:17,280
modeling on sort of the impact that this

64
00:02:14,760 --> 00:02:18,720
could directly Place Onto You by being

65
00:02:17,280 --> 00:02:20,640
the creator of this app with like

66
00:02:18,720 --> 00:02:22,860
because your GitHub name is out there

67
00:02:20,640 --> 00:02:24,599
how like your DNS creates that kind of

68
00:02:22,860 --> 00:02:26,760
stuff have you done anything to sort of

69
00:02:24,599 --> 00:02:29,040
cover yourself when it comes to building

70
00:02:26,760 --> 00:02:30,900
a tool like this that you know [ __ ]

71
00:02:29,040 --> 00:02:33,300
with too much time on their hands can

72
00:02:30,900 --> 00:02:37,260
just start and play with

73
00:02:33,300 --> 00:02:39,239
a little but only in so far as my GitHub

74
00:02:37,260 --> 00:02:41,220
account has also got 2fa on it most of

75
00:02:39,239 --> 00:02:44,700
most of what's important to me has

76
00:02:41,220 --> 00:02:46,620
reasonable protections on it my real

77
00:02:44,700 --> 00:02:48,480
name is of course everywhere

78
00:02:46,620 --> 00:02:49,860
um I am Tom Eastman that is actually on

79
00:02:48,480 --> 00:02:51,900
my birth certificate I'm findable

80
00:02:49,860 --> 00:02:54,000
someone could hack my run keeper and

81
00:02:51,900 --> 00:02:56,400
possibly find out where I lived there's

82
00:02:54,000 --> 00:02:58,800
there's interesting things once you

83
00:02:56,400 --> 00:03:01,739
become a Target yourself

84
00:02:58,800 --> 00:03:02,760
um I again get to okay I'm going to stop

85
00:03:01,739 --> 00:03:04,019
there but that's a little bit of

86
00:03:02,760 --> 00:03:04,860
foreshadowing we'll come we'll come back

87
00:03:04,019 --> 00:03:06,180
to that

88
00:03:04,860 --> 00:03:08,519
a little bit

89
00:03:06,180 --> 00:03:10,739
um don't don't worry about it it's cool

90
00:03:08,519 --> 00:03:13,140
um so what what is secretary secature is

91
00:03:10,739 --> 00:03:16,920
a little tiny web app it uses the

92
00:03:13,140 --> 00:03:18,900
Twitter API to allow someone to block a

93
00:03:16,920 --> 00:03:20,459
person who's harassed on Twitter and all

94
00:03:18,900 --> 00:03:23,940
of their followers for a set period of

95
00:03:20,459 --> 00:03:25,260
time so if someone is attacking you on

96
00:03:23,940 --> 00:03:26,159
Twitter and they have like 20 000

97
00:03:25,260 --> 00:03:27,780
followers and they're sending their

98
00:03:26,159 --> 00:03:29,340
followers after you

99
00:03:27,780 --> 00:03:30,780
um you can hit one button and you can

100
00:03:29,340 --> 00:03:32,060
block them and you can block all of

101
00:03:30,780 --> 00:03:34,080
their followers

102
00:03:32,060 --> 00:03:36,659
uh the

103
00:03:34,080 --> 00:03:38,760
it gave them it and it did it for a set

104
00:03:36,659 --> 00:03:40,739
period of time so um it kind of gave you

105
00:03:38,760 --> 00:03:42,659
two advantages over the usual sort of

106
00:03:40,739 --> 00:03:44,400
Twitter protection mechanism that they

107
00:03:42,659 --> 00:03:46,799
had it gave you the ability to

108
00:03:44,400 --> 00:03:48,299
neutralize a large group of people who

109
00:03:46,799 --> 00:03:49,860
are sort of trying to Dogpile you which

110
00:03:48,299 --> 00:03:52,620
is one of Twitter's

111
00:03:49,860 --> 00:03:54,599
great Innovations and harassment

112
00:03:52,620 --> 00:03:56,519
um and it gave you the freedom to do it

113
00:03:54,599 --> 00:03:57,959
fairly ruthlessly because you knew that

114
00:03:56,519 --> 00:03:59,640
like because you could block people

115
00:03:57,959 --> 00:04:00,480
temporarily

116
00:03:59,640 --> 00:04:01,799
um

117
00:04:00,480 --> 00:04:03,959
you knew that if you accidentally

118
00:04:01,799 --> 00:04:05,519
blocked nice people while you were doing

119
00:04:03,959 --> 00:04:07,019
it they would automatically be unblocked

120
00:04:05,519 --> 00:04:08,459
eventually and I found that to be a

121
00:04:07,019 --> 00:04:10,319
really useful tool

122
00:04:08,459 --> 00:04:12,659
um in my original talk on the subject I

123
00:04:10,319 --> 00:04:16,199
mentioned that Twitter allows you to

124
00:04:12,659 --> 00:04:18,540
mute topics for a set period of time

125
00:04:16,199 --> 00:04:19,620
um so they like have mechanisms inbuilt

126
00:04:18,540 --> 00:04:21,120
they understand the value of being able

127
00:04:19,620 --> 00:04:23,600
to protect yourself temporarily from

128
00:04:21,120 --> 00:04:26,520
like Star Wars spoilers but not from

129
00:04:23,600 --> 00:04:27,479
Nazis I don't know

130
00:04:26,520 --> 00:04:30,720
um

131
00:04:27,479 --> 00:04:33,000
it was a really unsophisticated UI this

132
00:04:30,720 --> 00:04:35,580
is basically the only uh interactive

133
00:04:33,000 --> 00:04:37,020
piece of the entire app uh this is what

134
00:04:35,580 --> 00:04:38,520
it looks like because I'm a pretty good

135
00:04:37,020 --> 00:04:40,020
python programmer I'm pretty good

136
00:04:38,520 --> 00:04:42,479
back-end programmer

137
00:04:40,020 --> 00:04:46,560
and I am just I'm just useless at HTML

138
00:04:42,479 --> 00:04:49,860
and CSS and ux it's just not what I do

139
00:04:46,560 --> 00:04:51,660
so for the python developers here

140
00:04:49,860 --> 00:04:54,180
this app wasn't a lot to write home

141
00:04:51,660 --> 00:04:57,660
about okay four Django models sort of

142
00:04:54,180 --> 00:05:00,180
mirroring some of what um Twitter's data

143
00:04:57,660 --> 00:05:02,040
model looks like a couple a half a dozen

144
00:05:00,180 --> 00:05:05,340
or a dozen generic Django views you know

145
00:05:02,040 --> 00:05:07,979
a list detail model detail uh literally

146
00:05:05,340 --> 00:05:09,960
one form as which you just saw

147
00:05:07,979 --> 00:05:12,300
um a dozen or so celery tasks and that's

148
00:05:09,960 --> 00:05:15,000
where some of the clever stuff happened

149
00:05:12,300 --> 00:05:19,620
um a crappy bootstrap UI

150
00:05:15,000 --> 00:05:21,840
um not a lot to it uh but it worked

151
00:05:19,620 --> 00:05:23,639
pretty hard and the salary tasks were

152
00:05:21,840 --> 00:05:27,120
probably the more interesting part I'm

153
00:05:23,639 --> 00:05:31,680
gonna run through uh an encapsulated

154
00:05:27,120 --> 00:05:33,900
timeline of how this came about uh I

155
00:05:31,680 --> 00:05:37,380
first had the idea

156
00:05:33,900 --> 00:05:40,560
um in 2016 at LCA actually

157
00:05:37,380 --> 00:05:42,660
um when I was talking to a

158
00:05:40,560 --> 00:05:45,360
online activist who had been involved in

159
00:05:42,660 --> 00:05:48,060
protecting people from gamergate

160
00:05:45,360 --> 00:05:49,979
um and I had the idea of like maybe this

161
00:05:48,060 --> 00:05:52,199
would be something that would be useful

162
00:05:49,979 --> 00:05:53,639
for people

163
00:05:52,199 --> 00:05:55,919
and then I didn't do anything with it

164
00:05:53,639 --> 00:05:59,100
for two years because ideas are easy and

165
00:05:55,919 --> 00:06:00,780
implementation is hard uh in 2018 I

166
00:05:59,100 --> 00:06:03,840
finally started writing code and I wrote

167
00:06:00,780 --> 00:06:06,300
it very much because I wanted it

168
00:06:03,840 --> 00:06:08,460
um Twitter in 2018 was just post

169
00:06:06,300 --> 00:06:10,560
Charlotte charlottesburg there was like

170
00:06:08,460 --> 00:06:12,660
a lot of ugliness there

171
00:06:10,560 --> 00:06:13,919
um and yet it was a place where because

172
00:06:12,660 --> 00:06:16,020
of my toxic relationship with social

173
00:06:13,919 --> 00:06:18,780
media I spent a lot of time there

174
00:06:16,020 --> 00:06:20,220
um I wrote it in Django but I wrote it

175
00:06:18,780 --> 00:06:22,319
for myself and that meant that it was

176
00:06:20,220 --> 00:06:23,940
largely command line interface it was

177
00:06:22,319 --> 00:06:25,740
really just local database and stuff but

178
00:06:23,940 --> 00:06:27,419
I knew that it would be a benefit for

179
00:06:25,740 --> 00:06:28,500
other people so I knew eventually I was

180
00:06:27,419 --> 00:06:30,000
going to get it online but that would

181
00:06:28,500 --> 00:06:32,220
involve

182
00:06:30,000 --> 00:06:33,419
writing HTML and writing CSS and you

183
00:06:32,220 --> 00:06:34,680
know the stuff that I don't like doing

184
00:06:33,419 --> 00:06:35,940
that much

185
00:06:34,680 --> 00:06:39,060
um

186
00:06:35,940 --> 00:06:42,060
most of the development for secatura

187
00:06:39,060 --> 00:06:44,699
happened in fits and starts and like do

188
00:06:42,060 --> 00:06:46,560
nothing on it for months and then like a

189
00:06:44,699 --> 00:06:48,720
Sprint of like a day or a weekend's

190
00:06:46,560 --> 00:06:51,120
hyper focused work or or a week's worth

191
00:06:48,720 --> 00:06:52,259
of work a lot of it was conference

192
00:06:51,120 --> 00:06:55,259
driven development which I'm sure

193
00:06:52,259 --> 00:06:57,000
several of you are familiar with so it's

194
00:06:55,259 --> 00:06:59,340
not a coincidence that it only finally

195
00:06:57,000 --> 00:07:02,220
got launched like a couple days before

196
00:06:59,340 --> 00:07:03,660
Picone Au 2019 because I was giving a

197
00:07:02,220 --> 00:07:05,520
talk on it and that's like oh crap I

198
00:07:03,660 --> 00:07:08,039
better actually finish this thing

199
00:07:05,520 --> 00:07:09,840
um it had its first couple users who

200
00:07:08,039 --> 00:07:11,819
weren't me by then a couple people who

201
00:07:09,840 --> 00:07:13,919
needed to protect themselves from uh

202
00:07:11,819 --> 00:07:14,880
transphobic attacks

203
00:07:13,919 --> 00:07:16,639
um

204
00:07:14,880 --> 00:07:19,259
and

205
00:07:16,639 --> 00:07:22,860
effectively people first started signing

206
00:07:19,259 --> 00:07:25,139
into it during my talk in uh at pikon EU

207
00:07:22,860 --> 00:07:27,199
in 2019. by the end of the year it had a

208
00:07:25,139 --> 00:07:30,360
couple hundred users

209
00:07:27,199 --> 00:07:34,919
in 2020 things went a little bit more

210
00:07:30,360 --> 00:07:38,400
nuts uh lockdown happened uh everyone

211
00:07:34,919 --> 00:07:41,900
was spending more time on Twitter I

212
00:07:38,400 --> 00:07:45,620
in about let's see is my mouse on screen

213
00:07:41,900 --> 00:07:48,720
in about May

214
00:07:45,620 --> 00:07:49,979
I stopped it requiring an invitation to

215
00:07:48,720 --> 00:07:51,120
join like you could sign up but then I

216
00:07:49,979 --> 00:07:52,919
would have to push a button to enable

217
00:07:51,120 --> 00:07:54,240
your account and I just let people sign

218
00:07:52,919 --> 00:07:56,520
up freely

219
00:07:54,240 --> 00:07:59,099
um and I also

220
00:07:56,520 --> 00:08:02,099
changed one feature which was if I go

221
00:07:59,099 --> 00:08:03,780
back to that form I added the forever

222
00:08:02,099 --> 00:08:05,819
button there because I thought that the

223
00:08:03,780 --> 00:08:08,099
key feature was the temporary blocking

224
00:08:05,819 --> 00:08:09,419
but I added the ability to let someone

225
00:08:08,099 --> 00:08:11,479
block forever

226
00:08:09,419 --> 00:08:11,479
um

227
00:08:13,259 --> 00:08:17,880
usage started to spread and started to

228
00:08:15,960 --> 00:08:20,460
get popular and

229
00:08:17,880 --> 00:08:21,780
the server that I was running it on

230
00:08:20,460 --> 00:08:23,879
um started falling over all the time

231
00:08:21,780 --> 00:08:26,720
because I was running it on a T2 tiny

232
00:08:23,879 --> 00:08:29,520
then it teaches small then a T2 medium

233
00:08:26,720 --> 00:08:31,800
I had I I go into a lot of detail on

234
00:08:29,520 --> 00:08:34,440
this in a in a different talk but um I

235
00:08:31,800 --> 00:08:37,140
spent a lot of time

236
00:08:34,440 --> 00:08:41,279
working on keeping it running on one

237
00:08:37,140 --> 00:08:43,620
instance of one Amazon device because I

238
00:08:41,279 --> 00:08:45,000
wanted the cost to be predictable when

239
00:08:43,620 --> 00:08:46,860
you think about a technical platform

240
00:08:45,000 --> 00:08:48,180
like this you can easily design it in

241
00:08:46,860 --> 00:08:49,500
your head such that it would be like I

242
00:08:48,180 --> 00:08:52,500
don't know you'd use lambdas to do the

243
00:08:49,500 --> 00:08:53,760
API calls and you'd use dynamodb and you

244
00:08:52,500 --> 00:08:55,080
could make it infinitely horizontally

245
00:08:53,760 --> 00:08:56,700
scalable and therefore infinitely

246
00:08:55,080 --> 00:08:59,120
expensive which is a bad idea for a

247
00:08:56,700 --> 00:08:59,120
hobby project

248
00:08:59,459 --> 00:09:05,160
um it crashed a lot in 2020 and so the

249
00:09:02,279 --> 00:09:06,360
talk that I gave uh in 2020 at paikon EU

250
00:09:05,160 --> 00:09:08,100
is probably the most technical one on

251
00:09:06,360 --> 00:09:12,480
the subject because I I learned a lot

252
00:09:08,100 --> 00:09:14,100
about tuning postgres and AWS gp2 iops

253
00:09:12,480 --> 00:09:15,899
exhaustion caused by disk rights from

254
00:09:14,100 --> 00:09:17,220
the database and RAM exhaustion swapping

255
00:09:15,899 --> 00:09:19,279
to the disk

256
00:09:17,220 --> 00:09:19,279
um

257
00:09:19,320 --> 00:09:22,440
but yeah like I thought I had solved the

258
00:09:21,060 --> 00:09:23,220
scaling problems I thought I thought I

259
00:09:22,440 --> 00:09:25,320
didn't think it was going to get too

260
00:09:23,220 --> 00:09:26,700
much more popular than that I'm going to

261
00:09:25,320 --> 00:09:29,220
just zoom the graph out a little bit for

262
00:09:26,700 --> 00:09:31,260
the context of where we were

263
00:09:29,220 --> 00:09:34,560
um

264
00:09:31,260 --> 00:09:38,880
in by the end of 2021 it had 30 000

265
00:09:34,560 --> 00:09:41,040
users so it it it kind of it's it's

266
00:09:38,880 --> 00:09:41,820
spread entirely by like word of mouth I

267
00:09:41,040 --> 00:09:46,620
think

268
00:09:41,820 --> 00:09:47,700
um and it was basically just doing quite

269
00:09:46,620 --> 00:09:48,899
a lot of work

270
00:09:47,700 --> 00:09:51,720
um I didn't do pretty much any

271
00:09:48,899 --> 00:09:54,480
development on it in 2021 I was too busy

272
00:09:51,720 --> 00:09:56,519
um by the end of 2022 it had 60 000

273
00:09:54,480 --> 00:09:59,700
users and it was making roughly six

274
00:09:56,519 --> 00:10:01,620
million API calls uh to Twitter per day

275
00:09:59,700 --> 00:10:03,060
uh by this stage it was running on the

276
00:10:01,620 --> 00:10:05,100
T4 medium you know the budget was

277
00:10:03,060 --> 00:10:07,560
starting to creep up a little bit

278
00:10:05,100 --> 00:10:09,000
um in April of that year I did like one

279
00:10:07,560 --> 00:10:10,920
of those hyper focusing Sprint things I

280
00:10:09,000 --> 00:10:12,060
took a week off work ostensibly to have

281
00:10:10,920 --> 00:10:13,980
a holiday and then I spent the whole

282
00:10:12,060 --> 00:10:15,660
damn week programming

283
00:10:13,980 --> 00:10:17,519
um you know I upgraded at the Django 4 I

284
00:10:15,660 --> 00:10:20,339
upgraded to postgres 14 I reworked the

285
00:10:17,519 --> 00:10:21,660
database code to make it more efficient

286
00:10:20,339 --> 00:10:23,100
um I bootstrapped the whole thing into

287
00:10:21,660 --> 00:10:24,240
open Telemetry and started sending data

288
00:10:23,100 --> 00:10:25,500
to Honeycomb which I'll talk about more

289
00:10:24,240 --> 00:10:27,240
in a moment

290
00:10:25,500 --> 00:10:28,620
um I finally realized that I was an

291
00:10:27,240 --> 00:10:30,720
idiot for running this thing in Amazon

292
00:10:28,620 --> 00:10:32,880
Sydney because like you know I live near

293
00:10:30,720 --> 00:10:34,920
Sydney well in Wellington

294
00:10:32,880 --> 00:10:36,300
um I should run things in Sydney it's

295
00:10:34,920 --> 00:10:39,000
spending all of its time talking to the

296
00:10:36,300 --> 00:10:41,820
Twitter API where's the Twitter API

297
00:10:39,000 --> 00:10:43,500
California so I finally realized that

298
00:10:41,820 --> 00:10:45,120
actually if you're if you're gonna build

299
00:10:43,500 --> 00:10:47,640
something you might as well put it

300
00:10:45,120 --> 00:10:49,260
anyway you cut a lot of costs by moving

301
00:10:47,640 --> 00:10:50,220
it to a graviton instance and putting it

302
00:10:49,260 --> 00:10:50,820
in Oregon

303
00:10:50,220 --> 00:10:52,920
um

304
00:10:50,820 --> 00:10:54,000
and I set up Django Waffle and I set up

305
00:10:52,920 --> 00:10:55,740
a patreon

306
00:10:54,000 --> 00:10:58,560
um

307
00:10:55,740 --> 00:10:59,820
and then finally in 2023 Twitter began

308
00:10:58,560 --> 00:11:01,320
if those of those of you who use Twitter

309
00:10:59,820 --> 00:11:02,940
probably know it seems to be in

310
00:11:01,320 --> 00:11:05,459
basically a cultural relevance death

311
00:11:02,940 --> 00:11:06,959
cycle uh it started letting all of the

312
00:11:05,459 --> 00:11:08,940
extremists back on Twitter while at the

313
00:11:06,959 --> 00:11:11,220
same time sort of ending its ability to

314
00:11:08,940 --> 00:11:12,480
deal with abuse uh and they announced

315
00:11:11,220 --> 00:11:15,240
that they'd be deprecating all of their

316
00:11:12,480 --> 00:11:17,339
free apis in a attempt to sort of get

317
00:11:15,240 --> 00:11:20,279
more money from developers

318
00:11:17,339 --> 00:11:22,800
um the free API was cut off I was

319
00:11:20,279 --> 00:11:25,140
suspended from Twitter on a sorry my

320
00:11:22,800 --> 00:11:27,740
developer credentials were suspended uh

321
00:11:25,140 --> 00:11:29,880
on Twitter

322
00:11:27,740 --> 00:11:30,839
mid-april one week after I started my

323
00:11:29,880 --> 00:11:31,800
new job

324
00:11:30,839 --> 00:11:33,360
um

325
00:11:31,800 --> 00:11:35,220
and

326
00:11:33,360 --> 00:11:36,839
every single login attempt an API call

327
00:11:35,220 --> 00:11:39,540
started returning 500s it was

328
00:11:36,839 --> 00:11:43,320
effectively dead so that was basically

329
00:11:39,540 --> 00:11:49,079
the life of the app from 2019 sorry 2019

330
00:11:43,320 --> 00:11:51,180
to 2023 it died anonymously at night uh

331
00:11:49,079 --> 00:11:51,899
woke him in the morning it was broken

332
00:11:51,180 --> 00:11:54,000
um

333
00:11:51,899 --> 00:11:56,120
end of talk not quite

334
00:11:54,000 --> 00:11:56,120
um

335
00:11:56,399 --> 00:12:03,000
that's the gist of it

336
00:11:58,860 --> 00:12:04,140
um I'm gonna give you sort of a grab bag

337
00:12:03,000 --> 00:12:06,240
of things that kind of happened along

338
00:12:04,140 --> 00:12:08,100
the way first of all

339
00:12:06,240 --> 00:12:10,320
just some numbers for a side project

340
00:12:08,100 --> 00:12:13,560
like this that got kind of popular it

341
00:12:10,320 --> 00:12:15,540
cost me about

342
00:12:13,560 --> 00:12:17,820
not sure if that number is right it cost

343
00:12:15,540 --> 00:12:20,040
me about six thousand dollars to run

344
00:12:17,820 --> 00:12:23,640
for its entire lifespan

345
00:12:20,040 --> 00:12:26,579
um and about a year into it like in 2020

346
00:12:23,640 --> 00:12:27,140
I started accepting donations

347
00:12:26,579 --> 00:12:29,959
um

348
00:12:27,140 --> 00:12:32,760
and the donations

349
00:12:29,959 --> 00:12:35,160
added up to just over six thousand

350
00:12:32,760 --> 00:12:36,540
dollars so I just barely didn't lose any

351
00:12:35,160 --> 00:12:37,260
money on it

352
00:12:36,540 --> 00:12:40,440
um

353
00:12:37,260 --> 00:12:43,800
it brought in yeah so over three years

354
00:12:40,440 --> 00:12:45,959
it grew to 70 000 users as you saw uh

355
00:12:43,800 --> 00:12:48,660
since early 2020 when I started logging

356
00:12:45,959 --> 00:12:50,820
these it made about four billion Twitter

357
00:12:48,660 --> 00:12:51,540
API calls total

358
00:12:50,820 --> 00:12:52,980
um

359
00:12:51,540 --> 00:12:55,740
according to their current pricing plan

360
00:12:52,980 --> 00:12:57,180
that would be pretty expensive

361
00:12:55,740 --> 00:12:58,620
um that'd be in the millions of dollars

362
00:12:57,180 --> 00:12:59,880
so it's one of the reasons why it's not

363
00:12:58,620 --> 00:13:03,060
really

364
00:12:59,880 --> 00:13:05,279
um if you're wondering the most blocked

365
00:13:03,060 --> 00:13:07,260
person on the planet using the tool is

366
00:13:05,279 --> 00:13:09,420
someone you've never heard of

367
00:13:07,260 --> 00:13:11,519
um it's someone who it's probably like a

368
00:13:09,420 --> 00:13:13,200
bot account because they follow like 200

369
00:13:11,519 --> 00:13:14,700
000 people so it stands to reason that

370
00:13:13,200 --> 00:13:16,500
like the person who got blocked the most

371
00:13:14,700 --> 00:13:18,180
by a tool that blocks the followers of

372
00:13:16,500 --> 00:13:20,040
people is someone who just seems to

373
00:13:18,180 --> 00:13:21,120
follow everybody on Twitter

374
00:13:20,040 --> 00:13:24,060
um

375
00:13:21,120 --> 00:13:24,959
one user on secretary used the site 8

376
00:13:24,060 --> 00:13:28,880
000 times

377
00:13:24,959 --> 00:13:31,680
so in their in their usage of the site

378
00:13:28,880 --> 00:13:34,620
they asked to block the followers of

379
00:13:31,680 --> 00:13:36,540
people 8 000 times total another user

380
00:13:34,620 --> 00:13:37,980
triggered 10 million API calls on their

381
00:13:36,540 --> 00:13:41,339
own so they they probably blocked a

382
00:13:37,980 --> 00:13:42,180
total of 10 million people on Twitter

383
00:13:41,339 --> 00:13:45,480
um

384
00:13:42,180 --> 00:13:48,540
so I'm going to talk a little bit about

385
00:13:45,480 --> 00:13:51,240
why this was good for me I guess

386
00:13:48,540 --> 00:13:52,680
um I learned a lot of stuff doing this

387
00:13:51,240 --> 00:13:54,540
um some of it I was already pretty good

388
00:13:52,680 --> 00:13:56,519
at and some of it I absolutely wasn't um

389
00:13:54,540 --> 00:13:58,200
I got a lot better at using redis for

390
00:13:56,519 --> 00:14:02,459
celery and quiche I learned a lot more

391
00:13:58,200 --> 00:14:03,899
about tuning postgres for performance

392
00:14:02,459 --> 00:14:05,519
um some pretty cool Advanced celery

393
00:14:03,899 --> 00:14:07,980
patterns because the hard parts of the

394
00:14:05,519 --> 00:14:11,880
celery work was um

395
00:14:07,980 --> 00:14:13,740
uh handling the rate limiting and the

396
00:14:11,880 --> 00:14:15,660
back off algorithms and just sort of the

397
00:14:13,740 --> 00:14:18,120
Fanning out of

398
00:14:15,660 --> 00:14:19,980
this call triggers 100 calls to get

399
00:14:18,120 --> 00:14:22,260
pages of followers which then trigger

400
00:14:19,980 --> 00:14:23,279
5000 tasks each to do the block the

401
00:14:22,260 --> 00:14:24,899
blocks of the followers and sort of

402
00:14:23,279 --> 00:14:27,000
tuning all that stuff

403
00:14:24,899 --> 00:14:29,399
um I got a lot better at Docker and

404
00:14:27,000 --> 00:14:31,139
Docker compose uh because that was the

405
00:14:29,399 --> 00:14:32,519
production platform was just a Docker

406
00:14:31,139 --> 00:14:35,880
compose file

407
00:14:32,519 --> 00:14:38,220
um running on an ec2 instance uh my pie

408
00:14:35,880 --> 00:14:39,839
for Django stuff struct log a lot of

409
00:14:38,220 --> 00:14:41,220
these things I then brought into my

410
00:14:39,839 --> 00:14:42,660
workplace

411
00:14:41,220 --> 00:14:44,100
um which was really handy it was like oh

412
00:14:42,660 --> 00:14:46,860
how did you know about this cool tool

413
00:14:44,100 --> 00:14:48,959
well I built it over here struct log

414
00:14:46,860 --> 00:14:51,300
open Telemetry honeycomb

415
00:14:48,959 --> 00:14:53,519
um psycho pg2 instrumentation and

416
00:14:51,300 --> 00:14:56,699
traffic for sort of load loading um

417
00:14:53,519 --> 00:14:58,500
sorry front-end load balancing stuff

418
00:14:56,699 --> 00:15:00,360
um okay

419
00:14:58,500 --> 00:15:01,380
I'm going to jump topics a little bit

420
00:15:00,360 --> 00:15:02,339
for the next little bit because I'm just

421
00:15:01,380 --> 00:15:04,500
going to talk about you know my favorite

422
00:15:02,339 --> 00:15:06,480
parts of what kind of happened here

423
00:15:04,500 --> 00:15:09,800
um the reason it stopped falling over

424
00:15:06,480 --> 00:15:14,160
dead was because I finally put user

425
00:15:09,800 --> 00:15:16,260
Centric rate limits onto secature um by

426
00:15:14,160 --> 00:15:18,480
far the biggest early mistake I made was

427
00:15:16,260 --> 00:15:20,820
not setting per user usage controls so

428
00:15:18,480 --> 00:15:23,519
that like this person over here

429
00:15:20,820 --> 00:15:25,800
who really needs the app because they're

430
00:15:23,519 --> 00:15:27,120
being attacked can't use it because this

431
00:15:25,800 --> 00:15:29,579
person over here

432
00:15:27,120 --> 00:15:31,260
has triggered the blocking of 15 million

433
00:15:29,579 --> 00:15:33,120
people and so it's going to take six

434
00:15:31,260 --> 00:15:34,740
days for that backlog to drain before

435
00:15:33,120 --> 00:15:35,579
coming over here

436
00:15:34,740 --> 00:15:37,740
um

437
00:15:35,579 --> 00:15:38,820
this is not interesting code but it's my

438
00:15:37,740 --> 00:15:40,260
favorite code on the whole thing because

439
00:15:38,820 --> 00:15:41,579
it was sort of simple and elegant and I

440
00:15:40,260 --> 00:15:43,019
got to use high school math

441
00:15:41,579 --> 00:15:44,459
who here has ever used high school math

442
00:15:43,019 --> 00:15:45,779
it's amazing

443
00:15:44,459 --> 00:15:46,800
okay some of you have used high school

444
00:15:45,779 --> 00:15:48,000
math fine

445
00:15:46,800 --> 00:15:50,459
[Applause]

446
00:15:48,000 --> 00:15:51,899
I I don't get to use high school math I

447
00:15:50,459 --> 00:15:54,120
was very excited it's like it's got a

448
00:15:51,899 --> 00:15:57,000
gradient you know like the the gradient

449
00:15:54,120 --> 00:15:58,620
thing and like anyway

450
00:15:57,000 --> 00:16:00,959
um the original the original account

451
00:15:58,620 --> 00:16:02,639
controls on the site where

452
00:16:00,959 --> 00:16:03,779
Tom has to enable your account first

453
00:16:02,639 --> 00:16:05,220
that's the thing that I got rid of

454
00:16:03,779 --> 00:16:06,959
pretty early you're not allowed to block

455
00:16:05,220 --> 00:16:09,420
all of your own followers because that

456
00:16:06,959 --> 00:16:12,660
would be a recipe for some pain

457
00:16:09,420 --> 00:16:14,699
um and I had to have a limit on like

458
00:16:12,660 --> 00:16:16,079
you can't block someone you can't block

459
00:16:14,699 --> 00:16:18,660
all the followers of someone with like

460
00:16:16,079 --> 00:16:19,980
over 500 000 followers because once you

461
00:16:18,660 --> 00:16:21,180
get into the millions it's just not

462
00:16:19,980 --> 00:16:22,440
practical with the tools that Twitter

463
00:16:21,180 --> 00:16:23,940
gives you

464
00:16:22,440 --> 00:16:25,740
um for example to block all of Donald

465
00:16:23,940 --> 00:16:27,000
Trump's followers you'd have to with

466
00:16:25,740 --> 00:16:29,220
Twitter's rate limits it would take

467
00:16:27,000 --> 00:16:31,920
about seven weeks just to download the

468
00:16:29,220 --> 00:16:33,360
list so it's just not really practical

469
00:16:31,920 --> 00:16:36,120
um

470
00:16:33,360 --> 00:16:37,920
but once I built this mechanism I was

471
00:16:36,120 --> 00:16:39,660
able to keep the site online by tuning

472
00:16:37,920 --> 00:16:42,000
this and that's far better than like

473
00:16:39,660 --> 00:16:44,519
trying to work out oh if I add more

474
00:16:42,000 --> 00:16:47,160
threads or if I switch to G event or um

475
00:16:44,519 --> 00:16:49,980
or or is it finally time to do the

476
00:16:47,160 --> 00:16:51,500
dynamodb Lambda thing

477
00:16:49,980 --> 00:16:54,360
um

478
00:16:51,500 --> 00:16:56,160
eventually with a lot of experimentation

479
00:16:54,360 --> 00:16:58,259
on these rate limits what I settled for

480
00:16:56,160 --> 00:17:00,480
was um

481
00:16:58,259 --> 00:17:03,779
if you first signed up to secretary you

482
00:17:00,480 --> 00:17:05,339
had a bucket and you and your bucket was

483
00:17:03,779 --> 00:17:07,559
full and you could block a lot of people

484
00:17:05,339 --> 00:17:09,720
with the tokens in that bucket and then

485
00:17:07,559 --> 00:17:11,760
the tokens refilled quite slowly so

486
00:17:09,720 --> 00:17:13,620
maybe you could block 200 000 people

487
00:17:11,760 --> 00:17:16,020
when you first join

488
00:17:13,620 --> 00:17:17,459
um but then from then on it only refills

489
00:17:16,020 --> 00:17:21,199
at the rate of like five thousand a day

490
00:17:17,459 --> 00:17:22,760
and that seemed to be a very

491
00:17:21,199 --> 00:17:25,500
sustainable

492
00:17:22,760 --> 00:17:26,760
use model for this

493
00:17:25,500 --> 00:17:28,319
um but all of that just comes from

494
00:17:26,760 --> 00:17:30,780
experimentation and what was really

495
00:17:28,319 --> 00:17:32,340
valuable was just having the ability to

496
00:17:30,780 --> 00:17:33,660
if someone came to me and said hey I'm

497
00:17:32,340 --> 00:17:34,559
being attacked by this person over here

498
00:17:33,660 --> 00:17:36,780
and they actually have a ton of

499
00:17:34,559 --> 00:17:38,280
followers can you please help

500
00:17:36,780 --> 00:17:40,080
knowing that there was capacity in the

501
00:17:38,280 --> 00:17:41,580
system for me to just say like

502
00:17:40,080 --> 00:17:43,380
anyone who ever asked me that I was able

503
00:17:41,580 --> 00:17:46,080
to go yep go for it I've just refilled

504
00:17:43,380 --> 00:17:47,460
your your rate limit just go nuts um

505
00:17:46,080 --> 00:17:49,980
and it meant that all the sort of

506
00:17:47,460 --> 00:17:52,940
drive-by background usage uh didn't

507
00:17:49,980 --> 00:17:52,940
bring the whole thing down

508
00:17:53,940 --> 00:17:58,919
I told you that I didn't do basically

509
00:17:55,919 --> 00:18:00,960
any work on the in 2021 and that was

510
00:17:58,919 --> 00:18:03,299
pretty much because

511
00:18:00,960 --> 00:18:05,400
it got really unwieldy and scary to do

512
00:18:03,299 --> 00:18:06,840
because the database by then was really

513
00:18:05,400 --> 00:18:09,660
big and it was on a really small

514
00:18:06,840 --> 00:18:11,039
instance and it was very difficult to

515
00:18:09,660 --> 00:18:12,419
test production conditions it was

516
00:18:11,039 --> 00:18:14,700
impossible to test production conditions

517
00:18:12,419 --> 00:18:15,720
and I thought that that meant that I was

518
00:18:14,700 --> 00:18:16,860
failing as a developer right because

519
00:18:15,720 --> 00:18:18,299
you're supposed to be able to do good

520
00:18:16,860 --> 00:18:20,640
testing do good unit testing do good

521
00:18:18,299 --> 00:18:22,200
integration testing do good load testing

522
00:18:20,640 --> 00:18:24,000
in practice

523
00:18:22,200 --> 00:18:26,039
that's really hard

524
00:18:24,000 --> 00:18:28,200
um and when I was trying to build new

525
00:18:26,039 --> 00:18:29,880
features or things for this a single

526
00:18:28,200 --> 00:18:31,740
sequential scan would basically bring

527
00:18:29,880 --> 00:18:33,240
down the whole server because the disk

528
00:18:31,740 --> 00:18:35,039
IO would be way too much for the tiny

529
00:18:33,240 --> 00:18:37,140
instance it was running on

530
00:18:35,039 --> 00:18:39,299
um and if you're using postgres as your

531
00:18:37,140 --> 00:18:40,620
backing store you don't necessarily know

532
00:18:39,299 --> 00:18:42,440
if you've built something that's not

533
00:18:40,620 --> 00:18:44,580
going to use a sequential scan because

534
00:18:42,440 --> 00:18:45,780
postgres's query plan or behavior

535
00:18:44,580 --> 00:18:50,120
changes depending on the size of your

536
00:18:45,780 --> 00:18:50,120
tables so when you are working from home

537
00:18:53,000 --> 00:18:58,020
when you're working on your local laptop

538
00:18:56,400 --> 00:18:59,820
with a couple hundred rows or a couple

539
00:18:58,020 --> 00:19:02,700
thousand rows postgres is going to

540
00:18:59,820 --> 00:19:04,140
behave differently to when you have 150

541
00:19:02,700 --> 00:19:07,440
million or 2 billion rows in your

542
00:19:04,140 --> 00:19:09,179
database uh so it's just very hard to

543
00:19:07,440 --> 00:19:13,559
deal with

544
00:19:09,179 --> 00:19:15,240
at LCA last year at the online LCA uh

545
00:19:13,559 --> 00:19:16,980
Liz Fong Jones gave a keynote address on

546
00:19:15,240 --> 00:19:19,620
observability engineering and it really

547
00:19:16,980 --> 00:19:21,360
opened my eyes to some cool new stuff

548
00:19:19,620 --> 00:19:22,380
um the first one was open Telemetry

549
00:19:21,360 --> 00:19:24,840
which I'm going to be giving a talk

550
00:19:22,380 --> 00:19:27,000
about at kiwi pycon next month which you

551
00:19:24,840 --> 00:19:29,039
guys should come to it'll be really cool

552
00:19:27,000 --> 00:19:29,760
um

553
00:19:29,039 --> 00:19:30,900
um

554
00:19:29,760 --> 00:19:33,660
I'm going to give a talk on open

555
00:19:30,900 --> 00:19:35,280
Telemetry there open Telemetry gave me

556
00:19:33,660 --> 00:19:37,679
the ability to see exactly where the

557
00:19:35,280 --> 00:19:39,720
slow and fast parts were in the running

558
00:19:37,679 --> 00:19:40,559
production code

559
00:19:39,720 --> 00:19:43,500
um

560
00:19:40,559 --> 00:19:45,600
but that's not the only topic that Liz

561
00:19:43,500 --> 00:19:46,799
discusses in in her keynote because the

562
00:19:45,600 --> 00:19:50,340
other aspect of observability

563
00:19:46,799 --> 00:19:52,260
engineering is the tacit admission that

564
00:19:50,340 --> 00:19:53,820
there's no such environment action you

565
00:19:52,260 --> 00:19:55,200
simply can't simulate production

566
00:19:53,820 --> 00:19:57,480
anywhere else and so you just have to

567
00:19:55,200 --> 00:19:59,460
learn how to safely develop on

568
00:19:57,480 --> 00:20:01,559
production she's talking about things at

569
00:19:59,460 --> 00:20:03,419
the scale of Google and stuff but it

570
00:20:01,559 --> 00:20:04,740
really counted for something like what I

571
00:20:03,419 --> 00:20:07,700
was dealing with where

572
00:20:04,740 --> 00:20:07,700
the um

573
00:20:08,580 --> 00:20:13,559
the constraints that I was under were a

574
00:20:11,100 --> 00:20:14,760
bit weird they were smaller but they

575
00:20:13,559 --> 00:20:17,160
were weird because the database was

576
00:20:14,760 --> 00:20:19,559
really big on a really small box

577
00:20:17,160 --> 00:20:21,240
um to be able to develop on production I

578
00:20:19,559 --> 00:20:22,559
put Django waffle in and started doing

579
00:20:21,240 --> 00:20:23,760
feature-based development where I could

580
00:20:22,559 --> 00:20:24,960
deploy it and I could roll it out to

581
00:20:23,760 --> 00:20:26,580
just myself

582
00:20:24,960 --> 00:20:27,660
make it work for just myself make it

583
00:20:26,580 --> 00:20:29,580
work for two people make it work for

584
00:20:27,660 --> 00:20:31,799
four then one percent of the user base

585
00:20:29,580 --> 00:20:32,760
then two then four then eight

586
00:20:31,799 --> 00:20:34,980
um

587
00:20:32,760 --> 00:20:37,440
it meant that I was able to get a crap

588
00:20:34,980 --> 00:20:40,320
ton done that I simply was too scared to

589
00:20:37,440 --> 00:20:41,760
do previously so I felt like that really

590
00:20:40,320 --> 00:20:45,059
changed what I was able to do and I've

591
00:20:41,760 --> 00:20:46,679
I'm a real sort of evangelist for it now

592
00:20:45,059 --> 00:20:49,679
um

593
00:20:46,679 --> 00:20:49,679
foreign

594
00:20:49,760 --> 00:20:53,299
built this tool for myself

595
00:20:53,580 --> 00:20:56,580
but I knew other people would want to

596
00:20:55,140 --> 00:20:57,960
use it but I was only half right about

597
00:20:56,580 --> 00:20:59,340
what people wanted being able to block

598
00:20:57,960 --> 00:21:01,799
all the followers of someone attacking

599
00:20:59,340 --> 00:21:02,400
you was incredibly valuable

600
00:21:01,799 --> 00:21:04,320
um

601
00:21:02,400 --> 00:21:06,480
but I thought the killer feature was the

602
00:21:04,320 --> 00:21:10,640
temporary block I thought that was

603
00:21:06,480 --> 00:21:14,419
absolutely what made the tool useful and

604
00:21:10,640 --> 00:21:17,400
nobody gave a crap about that everybody

605
00:21:14,419 --> 00:21:18,780
the overwhelming majority of the use was

606
00:21:17,400 --> 00:21:20,039
blocking people forever they just didn't

607
00:21:18,780 --> 00:21:22,320
care about the one thing that I cared

608
00:21:20,039 --> 00:21:24,120
about so I'm sure that a product manager

609
00:21:22,320 --> 00:21:26,000
type person will

610
00:21:24,120 --> 00:21:28,140
have a good lesson for you there about

611
00:21:26,000 --> 00:21:30,600
measuring what people want I don't know

612
00:21:28,140 --> 00:21:33,860
it I was I I built what I wanted and it

613
00:21:30,600 --> 00:21:33,860
was mostly what other people wanted

614
00:21:35,340 --> 00:21:38,039
it took me a little while to come around

615
00:21:36,659 --> 00:21:39,539
to asking for money but I'm glad I did

616
00:21:38,039 --> 00:21:41,159
because it didn't stay as cheap to run

617
00:21:39,539 --> 00:21:44,580
as I would have liked

618
00:21:41,159 --> 00:21:47,520
um I never intended to monetize the site

619
00:21:44,580 --> 00:21:49,740
I never wanted to like charge people who

620
00:21:47,520 --> 00:21:51,480
needed protection who needed the tool

621
00:21:49,740 --> 00:21:53,159
but what I really hoped was that the

622
00:21:51,480 --> 00:21:55,620
people who um

623
00:21:53,159 --> 00:21:57,780
who who could afford to donate

624
00:21:55,620 --> 00:21:59,760
would do so to support the people who

625
00:21:57,780 --> 00:22:01,380
couldn't it worked out but not really

626
00:21:59,760 --> 00:22:04,200
how I wanted I'm still a little bit sad

627
00:22:01,380 --> 00:22:06,900
about this but um well sad and grateful

628
00:22:04,200 --> 00:22:08,520
the overwhelming financial support that

629
00:22:06,900 --> 00:22:10,559
came to the tool was people like

630
00:22:08,520 --> 00:22:12,600
yourselves it was friends of mine who

631
00:22:10,559 --> 00:22:15,480
knew me it wasn't necessarily the people

632
00:22:12,600 --> 00:22:17,400
who were using the site so much

633
00:22:15,480 --> 00:22:19,080
um that was an interesting lesson I

634
00:22:17,400 --> 00:22:21,720
don't really know what to take from that

635
00:22:19,080 --> 00:22:22,440
uh apart from thank you

636
00:22:21,720 --> 00:22:24,539
um

637
00:22:22,440 --> 00:22:26,400
thank you this couldn't really have it

638
00:22:24,539 --> 00:22:29,340
couldn't have helped as many people as

639
00:22:26,400 --> 00:22:31,880
it did without the support of of the

640
00:22:29,340 --> 00:22:35,900
people who did financially support it

641
00:22:31,880 --> 00:22:35,900
but it was yeah it was

642
00:22:36,299 --> 00:22:39,659
it would have been like a lot of people

643
00:22:37,860 --> 00:22:41,520
who gave me good financial support

644
00:22:39,659 --> 00:22:44,940
didn't need the tool so I'm glad they

645
00:22:41,520 --> 00:22:47,000
supported it and yet I kind of yeah

646
00:22:44,940 --> 00:22:47,000
um

647
00:22:47,039 --> 00:22:50,159
I've I discovered that some of the

648
00:22:48,419 --> 00:22:53,780
weirder some of the use cases that

649
00:22:50,159 --> 00:22:53,780
happened just weren't um

650
00:22:54,860 --> 00:23:00,659
what I think even Twitter expected

651
00:22:58,380 --> 00:23:02,580
one thing that I learned that caused me

652
00:23:00,659 --> 00:23:05,159
a lot of problems with the site is that

653
00:23:02,580 --> 00:23:07,200
um there's a gigantic subculture on

654
00:23:05,159 --> 00:23:08,700
Twitter that are constantly changing

655
00:23:07,200 --> 00:23:11,880
their usernames

656
00:23:08,700 --> 00:23:13,320
like they put on and take off usernames

657
00:23:11,880 --> 00:23:14,820
like hats in the morning they're just

658
00:23:13,320 --> 00:23:15,900
constantly changing them and I get the

659
00:23:14,820 --> 00:23:19,440
impression that Twitter didn't really

660
00:23:15,900 --> 00:23:21,419
realize that either because uh

661
00:23:19,440 --> 00:23:22,620
the oauth credentials invalidate

662
00:23:21,419 --> 00:23:24,480
themselves every time someone changes

663
00:23:22,620 --> 00:23:25,799
their screen names so that would break a

664
00:23:24,480 --> 00:23:27,480
lot of the mechanisms on the site the

665
00:23:25,799 --> 00:23:28,500
unblock on Secretary of the unblocking

666
00:23:27,480 --> 00:23:30,900
thing

667
00:23:28,500 --> 00:23:34,340
um and it just was not a use case that I

668
00:23:30,900 --> 00:23:34,340
uh ever really expected

669
00:23:35,039 --> 00:23:40,500
so

670
00:23:37,080 --> 00:23:42,659
discussion that I had in 2019 about

671
00:23:40,500 --> 00:23:44,700
how almost all

672
00:23:42,659 --> 00:23:49,020
abuse protection mechanisms can be

673
00:23:44,700 --> 00:23:50,220
weaponized for abuse and I felt that

674
00:23:49,020 --> 00:23:52,220
this couldn't because I felt that

675
00:23:50,220 --> 00:23:54,720
blocking people was one of the things

676
00:23:52,220 --> 00:23:56,700
that couldn't really be weaponized

677
00:23:54,720 --> 00:23:59,220
against someone because it really is

678
00:23:56,700 --> 00:24:00,960
curating your own online experience

679
00:23:59,220 --> 00:24:03,240
and I'm pretty sure I was right about

680
00:24:00,960 --> 00:24:06,020
that but last year there was an

681
00:24:03,240 --> 00:24:09,419
interesting thing that

682
00:24:06,020 --> 00:24:12,260
there's there's an argument that could

683
00:24:09,419 --> 00:24:14,640
be made that it

684
00:24:12,260 --> 00:24:17,520
some people who I wish didn't use

685
00:24:14,640 --> 00:24:18,960
secretary used it for a benefit so I'm

686
00:24:17,520 --> 00:24:21,720
going to tell you about how my life got

687
00:24:18,960 --> 00:24:23,400
a little bit more surreal uh about a

688
00:24:21,720 --> 00:24:25,020
year ago

689
00:24:23,400 --> 00:24:27,240
um during

690
00:24:25,020 --> 00:24:28,919
the early days of the conflict of the

691
00:24:27,240 --> 00:24:32,000
Russia Ukraine war

692
00:24:28,919 --> 00:24:32,000
um there's a gigantic

693
00:24:32,039 --> 00:24:39,059
group of people on Twitter who are

694
00:24:35,280 --> 00:24:41,880
sort of harassing people who are

695
00:24:39,059 --> 00:24:44,039
spouting a lot of Russian propaganda

696
00:24:41,880 --> 00:24:46,679
um and

697
00:24:44,039 --> 00:24:48,659
I discovered this when I discovered that

698
00:24:46,679 --> 00:24:52,080
my tool secretary got really really

699
00:24:48,659 --> 00:24:54,299
popular among Russian propagandists

700
00:24:52,080 --> 00:24:56,700
for blocking the attacks from this other

701
00:24:54,299 --> 00:24:57,780
group of people I don't know if these

702
00:24:56,700 --> 00:24:59,940
are very readable but you don't really

703
00:24:57,780 --> 00:25:01,860
need to worry about it too much things

704
00:24:59,940 --> 00:25:04,679
started to get a little bit weird

705
00:25:01,860 --> 00:25:06,900
uh a lot of people were starting to

706
00:25:04,679 --> 00:25:08,820
notice that

707
00:25:06,900 --> 00:25:10,860
this group of people were using secateur

708
00:25:08,820 --> 00:25:12,659
and going okay well what's the deal with

709
00:25:10,860 --> 00:25:13,860
that tool who's who's made that what's

710
00:25:12,659 --> 00:25:16,200
going on

711
00:25:13,860 --> 00:25:18,299
um I felt a bit weird about this because

712
00:25:16,200 --> 00:25:19,620
like on the one hand I do not at all

713
00:25:18,299 --> 00:25:22,980
support

714
00:25:19,620 --> 00:25:24,960
the Russian invasion of Ukraine and on

715
00:25:22,980 --> 00:25:27,659
the other hand I built a tool that was

716
00:25:24,960 --> 00:25:30,240
designed to stop people from harassing

717
00:25:27,659 --> 00:25:32,159
people on Twitter and this group of

718
00:25:30,240 --> 00:25:34,860
people were you know harassment on

719
00:25:32,159 --> 00:25:36,659
Twitter so it was just

720
00:25:34,860 --> 00:25:38,100
people using the tool for what it was

721
00:25:36,659 --> 00:25:38,940
used for

722
00:25:38,100 --> 00:25:41,220
um

723
00:25:38,940 --> 00:25:43,080
at a certain point they decided to start

724
00:25:41,220 --> 00:25:43,980
looking into me

725
00:25:43,080 --> 00:25:46,559
um

726
00:25:43,980 --> 00:25:50,220
so I woke up one morning about a year

727
00:25:46,559 --> 00:25:51,659
ago and suddenly had a whole lot of

728
00:25:50,220 --> 00:25:53,159
things in my inbox about how I was

729
00:25:51,659 --> 00:25:56,159
helping the Russians and like who was I

730
00:25:53,159 --> 00:25:58,980
and was I a Russian spy uh

731
00:25:56,159 --> 00:26:02,340
was I was I an agent for the Russians

732
00:25:58,980 --> 00:26:05,340
things like that it got a bit weird they

733
00:26:02,340 --> 00:26:07,500
started digging into my websites and a

734
00:26:05,340 --> 00:26:08,820
few other bits and pieces

735
00:26:07,500 --> 00:26:11,340
um

736
00:26:08,820 --> 00:26:13,380
at one point they messaged some people

737
00:26:11,340 --> 00:26:15,120
from who claimed to be from Anonymous

738
00:26:13,380 --> 00:26:16,799
and those people sort of said oh look

739
00:26:15,120 --> 00:26:20,220
like this guy's clearly not a very good

740
00:26:16,799 --> 00:26:22,559
developer the SSL cert on his Blog has

741
00:26:20,220 --> 00:26:24,000
expired uh he clearly doesn't care much

742
00:26:22,559 --> 00:26:25,500
about security

743
00:26:24,000 --> 00:26:26,760
um I was staying way out of this because

744
00:26:25,500 --> 00:26:28,260
there was nothing that I could say that

745
00:26:26,760 --> 00:26:29,580
would like alleviate it either side

746
00:26:28,260 --> 00:26:31,440
because I thought either side was kind

747
00:26:29,580 --> 00:26:32,039
of problematic

748
00:26:31,440 --> 00:26:33,779
um

749
00:26:32,039 --> 00:26:35,220
but I really wanted to reply and say

750
00:26:33,779 --> 00:26:37,500
dude

751
00:26:35,220 --> 00:26:38,700
I get I get an email every 90 days from

752
00:26:37,500 --> 00:26:40,140
my friend Lee

753
00:26:38,700 --> 00:26:41,820
to tell me that my certificate has

754
00:26:40,140 --> 00:26:43,980
expired on my blog it's the only way I

755
00:26:41,820 --> 00:26:47,120
know anyone and anyone still reads it

756
00:26:43,980 --> 00:26:47,120
and so like

757
00:26:49,620 --> 00:26:54,720
things got weirder because

758
00:26:52,200 --> 00:26:55,740
on the Russian side they started saying

759
00:26:54,720 --> 00:26:58,320
hey how come we never heard about this

760
00:26:55,740 --> 00:27:00,360
tool until a few weeks ago maybe it's a

761
00:26:58,320 --> 00:27:04,860
false flag operation by the ukrainians

762
00:27:00,360 --> 00:27:07,200
to get rip addresses so and then so they

763
00:27:04,860 --> 00:27:09,360
started um

764
00:27:07,200 --> 00:27:12,059
sort of playing mind games with each

765
00:27:09,360 --> 00:27:15,059
other about whose side I was on at this

766
00:27:12,059 --> 00:27:18,720
point though I went and talked to uh ZX

767
00:27:15,059 --> 00:27:21,240
security who uh run kiwicon and koi Khan

768
00:27:18,720 --> 00:27:22,500
um and our friends of mine from

769
00:27:21,240 --> 00:27:23,220
Wellington

770
00:27:22,500 --> 00:27:24,900
um

771
00:27:23,220 --> 00:27:27,360
just sort of go okay well look if this

772
00:27:24,900 --> 00:27:29,520
becomes a bit more targeted like how

773
00:27:27,360 --> 00:27:32,640
much of my data really is online I was

774
00:27:29,520 --> 00:27:35,220
talking to Simon and he said well Tom do

775
00:27:32,640 --> 00:27:38,159
you still live on um Acura drive and I

776
00:27:35,220 --> 00:27:39,360
was like no but [ __ ] how did you find

777
00:27:38,159 --> 00:27:42,000
that

778
00:27:39,360 --> 00:27:43,200
um I had gone through like my link I've

779
00:27:42,000 --> 00:27:44,520
gone through my LinkedIn my run keeper

780
00:27:43,200 --> 00:27:45,840
all those things like strive I turned

781
00:27:44,520 --> 00:27:47,340
off all the maps I turned off all the

782
00:27:45,840 --> 00:27:49,080
friend sharing I turned off everything

783
00:27:47,340 --> 00:27:50,520
except you know made everything private

784
00:27:49,080 --> 00:27:53,039
mode for a little while

785
00:27:50,520 --> 00:27:54,900
um but it turns out that my phone number

786
00:27:53,039 --> 00:27:56,640
and my home address

787
00:27:54,900 --> 00:27:57,659
my old home address because I hadn't

788
00:27:56,640 --> 00:27:58,860
updated in a while was still like

789
00:27:57,659 --> 00:28:00,840
registered on a bunch of my New Zealand

790
00:27:58,860 --> 00:28:04,200
domains because New Zealand only

791
00:28:00,840 --> 00:28:05,640
recently added privacy protections to

792
00:28:04,200 --> 00:28:08,279
their who is database but you have to

793
00:28:05,640 --> 00:28:09,960
opt into them uh and I neglected to do

794
00:28:08,279 --> 00:28:11,580
so for some of my some of my personal

795
00:28:09,960 --> 00:28:14,159
domains

796
00:28:11,580 --> 00:28:17,159
um finally things got wow no we're

797
00:28:14,159 --> 00:28:19,100
almost to the end of the weirdness

798
00:28:17,159 --> 00:28:22,320
um

799
00:28:19,100 --> 00:28:24,299
some of the nepho people then said to

800
00:28:22,320 --> 00:28:27,179
the Russian people that I had spoken to

801
00:28:24,299 --> 00:28:29,400
the gcsb uh

802
00:28:27,179 --> 00:28:30,539
to hand over my database now I do want

803
00:28:29,400 --> 00:28:32,460
to just point out that the database just

804
00:28:30,539 --> 00:28:34,260
had public data in it anyway right like

805
00:28:32,460 --> 00:28:36,059
just but

806
00:28:34,260 --> 00:28:37,440
they started saying oh yeah I just got

807
00:28:36,059 --> 00:28:38,760
off the phone with Tom Eastman in New

808
00:28:37,440 --> 00:28:40,320
Zealand the developer of the tool and he

809
00:28:38,760 --> 00:28:42,840
has said that he is in touch with the

810
00:28:40,320 --> 00:28:44,820
special intelligence service

811
00:28:42,840 --> 00:28:46,740
um

812
00:28:44,820 --> 00:28:49,580
again like I just didn't reply to any of

813
00:28:46,740 --> 00:28:49,580
this stuff but um

814
00:28:51,299 --> 00:28:55,380
I I I worked in Security in Wellington

815
00:28:53,940 --> 00:28:56,760
and you have friends who work in

816
00:28:55,380 --> 00:28:57,900
Security in Wellington every once in a

817
00:28:56,760 --> 00:28:59,279
while they work for they start working

818
00:28:57,900 --> 00:29:00,360
for the government and you sort of oh so

819
00:28:59,279 --> 00:29:01,919
what department of the government do you

820
00:29:00,360 --> 00:29:04,799
work for and they get really cagey about

821
00:29:01,919 --> 00:29:07,380
it they're like just the government

822
00:29:04,799 --> 00:29:10,020
just the government and and so one of my

823
00:29:07,380 --> 00:29:12,299
friends messaged me one morning he just

824
00:29:10,020 --> 00:29:15,600
pointed to that tweet

825
00:29:12,299 --> 00:29:18,659
and he said Tom what the [ __ ] is this

826
00:29:15,600 --> 00:29:21,419
and I think the I think I replied with

827
00:29:18,659 --> 00:29:23,580
like a a face palm Emoji

828
00:29:21,419 --> 00:29:27,419
um but the the biggest irony of all this

829
00:29:23,580 --> 00:29:29,760
is yes because of this weird [ __ ] uh I

830
00:29:27,419 --> 00:29:31,620
did actually have someone from a

831
00:29:29,760 --> 00:29:35,340
government Department come and talk to

832
00:29:31,620 --> 00:29:38,520
me about uh they're like like

833
00:29:35,340 --> 00:29:40,860
he yeah he was like Tom what have you

834
00:29:38,520 --> 00:29:43,020
got yourself into like dude I don't even

835
00:29:40,860 --> 00:29:45,059
I don't even know

836
00:29:43,020 --> 00:29:46,679
um I'm almost out of time and I am

837
00:29:45,059 --> 00:29:47,760
almost out of slides

838
00:29:46,679 --> 00:29:50,940
um

839
00:29:47,760 --> 00:29:53,220
I wish I had like a point to all this it

840
00:29:50,940 --> 00:29:55,020
was worth it it was really worth it I

841
00:29:53,220 --> 00:29:56,159
helped a lot of people I learned a lot

842
00:29:55,020 --> 00:29:58,919
of things

843
00:29:56,159 --> 00:29:59,940
some [ __ ] got a little weird

844
00:29:58,919 --> 00:30:02,220
um

845
00:29:59,940 --> 00:30:04,260
I'm grateful to everyone who helped me

846
00:30:02,220 --> 00:30:06,120
do it uh

847
00:30:04,260 --> 00:30:07,260
I just thought I think one of the most

848
00:30:06,120 --> 00:30:11,640
important things that I learned though

849
00:30:07,260 --> 00:30:14,220
is that there's a there's a emotional

850
00:30:11,640 --> 00:30:16,860
and physical upper limit to how much

851
00:30:14,220 --> 00:30:18,600
effort that you can put into

852
00:30:16,860 --> 00:30:20,580
um to fixing someone else's trash fire

853
00:30:18,600 --> 00:30:21,779
when they don't want it fixed you know I

854
00:30:20,580 --> 00:30:24,779
put this effort into Twitter because

855
00:30:21,779 --> 00:30:27,659
people I cared about relied on the tool

856
00:30:24,779 --> 00:30:29,640
and were using it to communicate if I

857
00:30:27,659 --> 00:30:31,140
was in the same position today I would

858
00:30:29,640 --> 00:30:34,620
say no just get off the platform like

859
00:30:31,140 --> 00:30:37,080
just just it's like the the the the the

860
00:30:34,620 --> 00:30:38,880
degradation of the platform uh in the

861
00:30:37,080 --> 00:30:39,960
last year or so has been such that I

862
00:30:38,880 --> 00:30:43,140
don't think

863
00:30:39,960 --> 00:30:45,179
there's the an ability to protect on it

864
00:30:43,140 --> 00:30:46,679
um I'm out of time pretty much the one

865
00:30:45,179 --> 00:30:51,020
other thing that I wanted to discuss but

866
00:30:46,679 --> 00:30:51,020
I'm happy to talk about outside is um

867
00:30:51,720 --> 00:30:55,020
I put I put some effort into trying to

868
00:30:53,760 --> 00:30:57,000
keep it going after the credentials were

869
00:30:55,020 --> 00:30:58,080
busted there was a couple avenues that I

870
00:30:57,000 --> 00:31:00,480
went down

871
00:30:58,080 --> 00:31:01,860
um with credentials to like old apps

872
00:31:00,480 --> 00:31:02,820
that had kind of leaked and things like

873
00:31:01,860 --> 00:31:04,860
that

874
00:31:02,820 --> 00:31:06,299
um and for reasons it just wasn't going

875
00:31:04,860 --> 00:31:09,600
to work

876
00:31:06,299 --> 00:31:12,240
um and finally this happened today which

877
00:31:09,600 --> 00:31:13,679
kind of tells you this this this gives

878
00:31:12,240 --> 00:31:15,000
you a good clear indicator of where the

879
00:31:13,679 --> 00:31:17,159
future of this platform is going anyway

880
00:31:15,000 --> 00:31:18,840
right so

881
00:31:17,159 --> 00:31:20,640
thanks everyone who helped me out with

882
00:31:18,840 --> 00:31:22,080
this thanks everyone for listening now I

883
00:31:20,640 --> 00:31:24,659
can talk about this for hours like any

884
00:31:22,080 --> 00:31:26,039
any slide that you saw here I I love

885
00:31:24,659 --> 00:31:28,559
talking about this stuff there's a lot

886
00:31:26,039 --> 00:31:33,539
more that I learned

887
00:31:28,559 --> 00:31:35,940
um but yeah thank you for uh letting me

888
00:31:33,539 --> 00:31:38,480
thank you for listening

889
00:31:35,940 --> 00:31:38,480
thank you