Thoughts On BrandGhost Migration To Blob Storage

Transcript is auto-generated and may contain errors.

all right heading into the office uh it's rainy today so big surprise for Seattle area but uh it says it's going to be like a 50-minute drive which I'm really hoping is not the case because that sucks cuz it's already after 9: it's like 20 after 9 UM usually they get into the fast lane or whatever and I can cut down like 20 minutes so I'm I'm hoping that's what it is today but given that it's rainy even though it always rains here I feel like people are going to be worse at driving today just kind of expecting the worst on that but we'll see so this morning I was supposed to do a a podcast interview and uh the guy never showed up unfortunately this is probably not with this individual but probably the fourth or fifth time this has happened to me which

is really unfortunate um I I suspect in this case like you know what happens like you schedule these things out for a while and if you're not if you're not doing this kind of thing actively like I need to be looking at my schedule to see when I have podcasts and stuff coming up to record if you're not actively part participating or or hosting them or whatever then it's super easy to just be like oh crap like I forgot about it um so it happens uh but yeah it's kind of unfortunate um I didn't I didn't really miss out on much this morning because of it which is fine so last night I had a I did a podcast with someone who was traveling and they were in in France so they were in Paris I think and uh so I took that one pretty

pretty late um so not like super late it was like 10:45 uh so I got to bed just after midnight which is totally cool um but I meant like because I've been doing CrossFit for at 6 in the morning like I've been waking up at 5:00 and if I'm going to bed at midnight I'm not waking up at 5:00 to go to CrossFit so uh kind of planned it so that I would do that one and then sleep in a little bit and then do this other podcast for this morning uh but like I said unfortunately that didn't pan out so um didn't didn't miss the gym because this podcast uh got cancelled but uh so I was already at home whatever um so try to chip away at a couple of brand ghost things so like one thing that we're trying to do uh

from touch on like some technical details I guess so when you're posting stuff to social media depending on you know your experience with posting to different platforms generally the way that you do this is like in your app or on a web page or something right you if you want to post media you open up a browser on like a file browser on your phone or you know file folder browser on your computer go select whatever you want or you drag and drop it whatever but you have the exact file or files now that's great that's a pretty like common user experience when it comes to apis though for posting to social media they kind of there's like two major flavors that I see um and one is that you you tell the social media provider so like Instagram or Facebook Twitter Tik Tok YouTube

whatever you tell them when you're making uh an API call here is the URL where my resource is and then their server when it's handling the request will go pull from that URL right okay so pretty pretty straightforward it's just not the traditional way that you would do it if you were using the application directly now these social media providers also like almost always provide the alternative approach which is like a a streaming API so you will you have the btes for the media that you want and then you will be streaming those over to the server um so it's like two major flavors right you give it a URL or you are streaming bites to their server now the streaming approach also has a couple of like subf flavors and one is just like you basically uh start streaming bites and you just keep

writing while you have this connection open um and that seems like I I haven't written these social media apis right I've never been on the other side of it so I haven't put too much thought into this but that seems like it's kind of like a little bit more of like a legacy approach and I suppose the idea behind that is like the again I'm speculating here but I suspect it's because as media becomes larger and larger so you know maybe before when we were uploading videos to like Facebook it used to be like you'd have smaller videos not only in terms of length but file size so if it failed like whatever you just you'd redo it but now um now you'd be uploading content that's a lot bigger and if you were to just stream it kind of like a naive implementation of

just like take the bites and keep writing them over if you lost connection you just have to redo the whole thing so you're in the same situation as before right but with larger media I suspect they said hey this is kind of like a usability issue I'm sure there was uh data collected or whatever to get people to kind of switch to a different style here but this second flavor is like you are responsible for for chunking up the data that you send over so usually how this looks and again it's it's kind of different between platforms is sometimes you need to tell the other side here's how big my media is and I'm going to be sending over chunks of this size and then you you make different requests with each part of the chunk and this ideally gives you like some type of

like resumable concept so if you happen to lose connection and you needed to go re-upload data you you could write your client such that it could go restart the upload you know or sorry restart it from like a the last point where it had messed up type of thing so I think that's a lot of these have kind of moved that direction I could be wrong like I said I'm speculating so there's a handful of different flavors um now to kind of explain our technical side of things the easiest thing to get up and running as a platform is just having fewer things going on so for us it's like if we don't have to worry about hosting f FES and stuff it's pretty convenient um it's just one less thing to build so we started everything in brand ghost by having URLs to your

media so if you want to post text super easy the text is obviously just in our platform whatever we'll store it in a database but if you have media that you want to post or like we don't want to be in the business yet of of Hosting media like that's a that's just a whole other set of challenges to go build for but now we're at the point we like just expecting that people are going to host all of their own content is a kind of a like a usability problem and you kind of got to think about the evolution of some of these things right so with with brand ghost modeling it after how I've been building things out I have all of my content hosted somewhere right I I use had been using a system system that was kind of ad hoc before

brand ghost that needed to be able to access my media and now brand ghost as we've been building it follows the same pattern it's like tell me where your files are I will go pass them along to um sorry I just was trying to read a message while driving and vlogging so popped up as a notification so brand ghost will currently ask for the URL oh man this traffic is awful um and that's just modeling it after the existing Behavior so that's that's fine but we are reaching this point where it's kind of awkward that if we're onboarding people's content and we're like oh hey by the way like I don't know what you were doing before but go make sure that all of your media has public URLs to them so that we can access them because what we're doing behind the scenes is

taking the approach where if we have a URL we can either just give that to the social media provider so they can do the streaming right offloads the downloading on our server or or we basically will on our server stream chunks of your media and like basically be the intermediary where we're streaming the chunks and then then writing them out to the server so we don't have to download a whole file ever uh but we would allocate some memory to be able to like take the chunks and then and send them over as chunks so we need to be able to start hosting people's media uh which is a you know it's a solved problem but we have to go build it so uh that's kind of what I'm starting to look at now is like like Azure blob storage uh if you're not familiar

with Azure blob storage then like um like S3 buckets and AWS same idea this is so bad the stretch of highway says there's literally 15 minutes of uh of traffic added um on Google Maps and like I was saying this the other day I'm in the fast lane I can guarantee I'm paying cuz the price varies I can guarantee I'm paying Max price because it's slow and we're I'm going to be stopped in a moment like like I just was I was l parked in the fast lane paying money to be here so I'm not I'm already not having a great morning so I'm not super stoked but um yeah the the Azure blob storage part um I've done it with uh AWS so with meal coach we were designing features where you could upload your progress picture so your coach would be able to

have access to those uh so we' we've built that um to be able to upload we've been we've done that so that the client can can pull down uh the pictures as well I've done a couple of things like this even for my blog uh again with this is with Amazon but um I was pay Walling some content on my blog at one point and the way that the plugin for WordPress worked was that I could upload my content to S3 and then um when people needed to access the page they would have like basically a a time limit to start watching the video and then that would get streamed down to the browser so um the idea there was that your stuff is in Azure blob storage and you can kind of do or sorry in that case it was S3 so it's in

some type of blob storage whether it's AWS I don't know what Google's is uh Azure and you have binary data there and you now need a URL to it and you can have like different clients for this type of thing so depending on the client they might I don't know like abstract all this away and you can just get like a streaming API to it or download and get a bite array like whatever it happens to be um but in some cases and it depends on the situation right in some cases you might say I need a URL to this file so you create a situation where like I guess you might call it like a lease or like a Time expiry URL where you ask the uh the The Blob storage again whatever platform you're using you're saying for this blob I would like

a URL to it and I would like and then you can kind of add these constraints like I'd like it accessible uh from you know people offed with this or it's public and uh it will expire after X period of time so you can configure these different things now for us I'll have to figure out like on our server side if that even matters um like we'll we'll make it off let's say but I'll need to figure out on the server side and maybe I need to support multiple variations but do I need to be able to get a URL or uh do I need to stream it because there's some platforms I told you the URL versus like chunking approach um those are the two major approaches I see across these but I don't think all of them support both and I say that

without having the list of them in front of me and like the code in front of me but I say that because I had to implement both variations throughout this process so far so I think some of them uh have both and some have one or the other and I think that's going to mean that if we're using blob storage I will need supporting of uh of URLs so given a blob give me a URL to it I will take that URL and give it to the social media provider and then the other approach is uh given some blob resource like let me start streaming it because I now need to go take those chunks and stream them to the social media providers so I think I'll probably need both um so you got to build that but here's the other thing like it's not

just as simple as having the content in Blob storage because now you have to think about the the user experience for that right so um easy for me is one of the developers on this because for my own content like a lot of the time I skip the uh the user interface because I'm already in the code and I'm building out how I want to post my social media content like by directly talking with the database and this kind of thing so uh when I'm on the go I absolutely you know use uh the website on my phone and I'll I'll I'll write up some posts and stuff like that but uh if I'm creating stuff from scratch sometimes I'll just uh like a whole topic stream to go post sometimes I'll just do that directly in code because like I have it all open

on my computer anyway so the user experience is going to be kind of weird well sorry I don't mean the resulting experience will be weird hopefully it's not but kind of a an interesting challenge we have to think through because we probably need to build like a like a a file explorer right for our content or sorry for your content that we are hosting for you so if you upload a picture and you want to use it in a topic stream or for a piece of content in your topic stream you probably want to be able to see it select it and say use this picture but like what happens when you have multiple pictures so now you have 10 pictures 100 pictures when you're adding content you probably want to be able to see that now alternatively what this could look like cuz again

I have to think about the ux and we haven't we haven't built it yet alternatively maybe all that it is is when you're creating a Content entry in brand ghost so you're saying okay I have a topic stream that posts these days these times and it's to be picture posts you have that created now you're saying I want to add a new piece of content to post directly at the time when you want to go create that piece of content then you could go upload the picture and then only when you want to go um modify your content or whatever or view your content that's when we would pull that down from uh from blob storage through our server uh either as a URL to give to the front end probably uh just to save on on downloading bytes so we might not even need

like a browser for it so even if you wanted to use the same picture we might just uh we might just basically not show you a browser for it at all so if you needed two pieces of content that were identical for the picture you just have to upload it again that might be like the most trivial way to do it least amount of overhead because going to build like a file explorer and then manage your pictures and all that like that might be a pain in the butt but then that makes me think uh again maybe this makes it easy when you delete a piece of content so you're like I don't want to post this post to like LinkedIn and Facebook anymore like get rid of it that if we know that it's associated with a picture for example if we know that

only one uh piece of content in our platform can be mapped to that binary resource then we don't have to worry about oh is anyone else using this is it okay we would just say delete it from blob storage as well so you know maybe just trying to hyperfocus on Simplicity is the way to go here so um I have to go through this thought exercise with the team and make sure that we can think about the right way to do it but it's kind of what I'm thinking at a high level I was red cuz I saw another message come in the guy who uh who missed the podcast this morning he sent me he just sent me a message and it popped up on my phone I think there was a problem with the invite or something I'm I'm not going to read

it while I'm driving but I just saw it pop up so um that sucks but I I I hate being in this position now or this happens all the time uh I'm not saying I've never texted while driving of course but um especially in a situation like now where there's traffic and it stop and go like not going to be pulling out my phone for anything um but I hate when I have seen something like this because I'm like I want to respond to him to make sure we can just you know uh connect and be like hey you know it's all good and figure out next steps but like I'm not doing that while I'm driving so now I have to hold it with me um which sucks but that's okay so according to Google maps in like 10 ft the traffic stops apparently

there's an accident but we'll see I don't believe it to be honest I bet you it's going to go further oh you know what no no no it's it's trying to tell me to get off the highway cuz it's going to be faster but um the problem is that it's hard to trust Google Maps for this highway for timing and it's because it doesn't factor in the fast lane so if there is an accident and it's slowing everything down um it doesn't always affect the fast lane so sometimes it's no issue at all for the fast lane actually most of the time it's no issue at all and then there's some rare exceptions like right now where it's awful cuz like what it's been it's been 20 minutes already I don't I don't know if I'm halfway to work yet so just stupid but they

really like I'm going they got to do something about this road yeah like good luck um but this is why driving to the office sucks there was a comment on one of the last videos and it was like oh at least you get to drive a a nice car and go fast and it's like sometimes not when you're sitting in this stupid lane going 30 m an hour um can't go very fast I can go loud but not fast um but yeah that's how it is so yeah brand ghost will be doing um The Blob storage stuff next that one that's going to be a really helpful feature um it's funny I'm not like super excited to go build it cuz it's like it doesn't feel like it's progressing usability forward at all like it's not like a a new feature for the end user

like so that's not exciting but I know that it's going to help us um part of the other reason I'm hesitating is like uh a lot of how the content is managed right now relies on those URLs for posting and I need to go figure out if I need to switch everything uh to being like media resource IDs uh if I need to have a hybrid approach it's funny the car car put up the the flat tire warning just now what else could go wrong today but it's like the the PSI is like one off so it's kind of funny cuz it shows which Tire it's like the back uh passenger side tire is is low it says low not flat so I I apologize for for misleading but yeah it's like one PSI lower than the other Tire like oh um that's fine whatever

it's probably cuz it's cooler today so blob storage um we're having we got to get our Tik Tok stu stuff approved um so I think I mentioned this in one of the other uh drives in or drives back home but the challenge with Tik Tok this guy you can't cross that what are you doing you can't do that man you didn't see it but that was the illegal cross not cool do it in front of someone else not me um lost my train of thought so oh man brand goes Tik Tok there we go so Tik Tok basically approved our app like instantly it was like 4 hours and we're like hell yeah because meta was over a month and we had to go make videos and and stuff but then we got it approved and nothing was working still so I'm like well what

the hell's going on here and I researched a little bit more and now you have to get individual permissions approved just like Facebook so we felt duped um so right now we just have to get that done um so one of the other guys has been doing all these submissions but uh in order to make that happen uh we just have to correct something in our o it's kind of funny the the ooth is telling us that there's like there's no uh expiration date and maybe that's the case maybe there actually isn't an expiration date for the the tokens on uh on Twitter or on Tik Tok sorry I kind of doubt it but it's possible so we have to go check that um because we have an assumption right now on our code H that's that's wrong and basically it makes it look like

you're expired as soon as you authenticate so we just got to clear that up which I will be a very very quick fix um I just need someone who's been working on the front end to be able to tell me what data is there and we're good and then uh we'll submit to Tik Tok to try and get an approval so that'll be good get that out of the way because all the apis work and stuff we've well that's not fair to say we've tested them it makes post to Tik Tok but the videos don't they're just like empty which doesn't make sense but I've confirmed we're sending the right bites over so I have I don't know if it's just because um because it's in like sandbox mode if it's like we're not actually going to post your videos like that would just be

a waste of resources or something I I don't know so hoping that once we get the approval there's no other stuff to mess around with uh so crossing my fingers on that Google was really quick to approve us for YouTube but they have these stupid quota limits I talked about this before for sure but it means we can only allow across all of our users to post six videos per day which I'm already going to have at least one of those one of the six and I'm not even one of our end like real end users so that's not good so we request a quota bump but that's been a couple weeks now so we got to get ahead on that because we can't turn can't really turn YouTube on for other people until that's done so Tik Tok and YouTube are very close to

being added to the platform um I had uh someone from my my Discord Community was asking about uh kind of like some project ideas and I had mentioned like if you they want like something we need is going to be like Tumblr posting Reddit posting so they're they're playing around with that uh which is super cool I told them like and I like being transparent about this like we're not in a position where we can pay anyone for stuff we're not making money so it makes sense that we would just go build these things when we're ready like Tumblr and uh and Reddit are lower on our list of things to prioritize like we could totally survive without those um so I told this individual this is what I'm working on uh these are things coming up for us at some point so if you

want to I these would be fun things to try however there's no obligation that you like need to share your code with us like if you just want to do it for the exercise by all means please do that because I don't want someone to feel like they don't owe me anything um and I don't want anyone to feel that pressure so if they want to share their code they got it working awesome if not still awesome I'm glad that they're able to go and and learn something about it cuz I don't think people realize but like ooth which is I guess like the standardized way to do authentication just sucks it's like every single provider is doing it differently and it's infuriating it's kind of like uh when I was doing digital forensics we wanted to do like uh like basically like Bluetooth device

forensics like in a general way so I was prototyping something and Bluetooth is a standard but it turns out people do whatever the hell they want another fun fact zip file format is a standard you would be completely blown away by the number of like tools that make nonconforming zip files and the number of tools that just happen to support those non-conforming zip files so you have zip files that get created in different tools that are technically not right and then you have tools that support these technically not right zip files so people don't even realize that they're using these zip files that are just like not good like the file format's wrong uh and then you don't realize until until you're putting into another tool and it's not working so standards are good um but like they're only good if they're being enforced so

it's a pain so ooth I think as much as I hate it it's a good experience for people to go through just to understand like it's uh it's just not straightforward as much as I guess they tried to to make it that way so yeah we'll have more Platforms in brand go soon uh onboarding stuff uh I I think the last piece it's missing is I just have to make a a backend API for our front end to call to do like a bulk import and we already so we have um we have an API that works like this already and it's very developer Centric like I call it on Postman and uh uh it's not really meant for the front end to use because I don't know how the front end was ever going to use it and I thought that they were asking

me if uh like if that was ready to go and I was saying well I need you to tell me like tell me what you need from the front end perspective and I'll I'll tweak it because like I said it's kind of just been something I designed to to import stuff off my desktop and uh I I realized this morning when I was talking the engineers are like no no no like we don't need that API we need the the API that that creates a piece of content that already exist we just need that to support many entries and I said oh that's that's so much easier uh so much easier it's an array instead of a single item and then a for Loop somewhere else so that'll be good um I'll get that done tonight oh no oh no my God you see this

this truck this beside me all the wood on it I said this the other day about people getting in front of trucks right I just watched a Tesla go in front of that big truck that just passed me with had all these sheets of wood on it it's like two trailers long and I saw a Tesla get in front of that truck and then try to merge into the lane in front of me uh and there's no spots so instead of getting over he's in front of this truck just breaking and I watched the truck come up and I I didn't think he was going to be able to stop so that would have been pretty bad worked out though that's good news so this might be I have to see um this might be the last day drive into the office this week um

I don't think there were other folks that were flying in that I need to get a chance to talk with but I I thought my PM that I've been meeting with had an offsite today and I I think that I confirm that's not the case I think they just said that they were go sorry I meant for tomorrow so I thought he was going to be out tomorrow at an offsite but I think he was saying it's just like a lunch or something so uh I might might still come in the problem is that that I do a lot of one-on ones on Thursdays so might be kind of silly for me to drive in and just be locked in a room and be on calls the entire time and I have a lot of remote employees that I do uh one-on-one calls with on

Thursday so I don't know I might not I might I might not drivve in CU it's not going to be worth it for me like literally just wasting an hour and a half to two hours just I mean I'll hang out with you on on this but I feel like not the best use of my time come on Porsche go faster you're killing me you're killing me anyway having a having a sour day so far I'm just not not in a good mood I have to turn that around for getting into the office though it's easier to be in a bad mood at home because before I get onto a camera I can just I can I can put it on um yeah just I don't know kind of a kind of a lousy morning and not because the podcast thing like that happens right

it's just like that's a tiny tiny little thing and uh I don't know I just feel like this morning I wasn't having a great morning so feel like agitated I guess is a good way to put it and the problem well it might be obvious but I'll just say but the problem when I'm feeling kind of agitated is that everything keeps like perpetuating the agitated feeling um usually how I overcome that is like if I'm if I'm agitated in the morning and I'm working from home if I take a little nap I just reset so you know over lunch I'll take a you know whatever I only need like 8 to 15 minutes to nap cuz I fall asleep very quickly and uh that could be enough just to reset my mood sometimes so I can't well I don't want to say can't do that

at the office I probably could people might look at me funny but um so anyway I'm kind of like I don't have a I don't have a parachute to pull for my bad mood but I'll uh I'll try my best I think something that's like I've been practicing for a long time right is like I very introverted you might not know this but because you're like hey well that doesn't seem right considering you're talking to a camera and vlogging and you're on YouTube and stuff but like it's different um like being around people takes a lot of energy for me and uh for example like tomorrow on my 101 days that are really heavy at the end of the day I feel like I need to sleep immediately um and I always try to be clear about this that's not to say like I don't

like the people I'm talking to that's not the case I really like my team um it just takes a lot of energy from me uh so going to the office is kind of similar but um I can't remember what I was trying to say exactly I was exiting the highway here um and kind of lost my train of thought but yeah it's a lot of energy uh introverted oh turnning yeah like this IDE of like practicing like when I'm managing people and I've been doing this for like 12 years now I don't like I know that I'm introverted but I I don't like I don't like leaving people in a situation where they're like stuck with someone who like my natural state would be to be socially awkward like it's like I'm not going to talk and I don't think that's okay uh in settings

like one-on ones and and things like that so I kind of have to turn on more of an extroverted um Persona if you will right like it's not fake like I'm still me but I have to like I just have to put a lot of effort into not uh not like curling up into a ball and sitting in the corner just because it's uh exhausting so I feel like I should be okay to not be a jerk to people I like to think that and uh the side effect of that though is like if I'm not in a good mood and I have to kind of like make myself be presentable in a in a good mood then that's going to Tire me out a ton so we'll see the reason that's exra challenging today uh I don't know I haven't talked about my caffeine

Journey too much maybe uh if I remember on the way back I'll I'll talk about this cuz it's kind of interesting but um I don't drink coffee but I have been very big on caffeine for a very long time and I still am uh but I don't drink coffee I have caffeine powder the problem is that uh we finished all of the flavoring that I use for water when I add caffeine to it we finished it uh like one day before our Amazon order shows up for more flavor and and not a big deal for my wife because she likes coffee but it's a big deal for me cuz I don't and I need caffeine so I have this little we had one thing that was flavored and I have orange juice um so that's that little thing of orange juice I'll hold it up

again just cuz it's funny but um this thing of orange juice has 400 migs of caffeine in it so uh Monster Energy drinks in most places outside of the US I think have like 143 migr of caffeine or maybe it's just in in Canada they have 143 Mig of caffeine and then like bang energy and Monster energy drinks um in the US a lot of them have 300 Mig of caffeine so this little thing of orange juice has more caffeine than one of those drinks um I think a little can of Red Bull the standard one is like 80 Mig so it's five Red Bulls in that little thing of orange juice um and that's kind of like my current uh limit that I like to consume this is going to be one of the worst parking spots but at least no one can ding

my door on the other side you might see in just a moment what I mean oh my goodness goodness okay look at this there a wall I don't I don't want to talk badly about people at Microsoft but I want to talk about the parking situation because I have never seen such bad parking in my entire life now I can tell ahead of me there's a car who did this very smart sometimes people will park way over the lines but they're doing it in a situation where the space is not usable so for example there's a Mustang in front of me who is parked way over the line but what he's done or he or she has done is put their passenger door against the wall basically so they can open their driver side door and it's a coupe so the doors are very long

there's other times where I'm walking through and it's in this parking lot in particular it blows my mind I see people that have just pulled into spots random spots and they're just not in the lines and I can't comprehend how badly some of the parking is so I'm sorry if you're parking at this building there's statistically a lot of really bad Parkers now this spot's challenging for me cuz I got less than a foot and I have these long coup doors so it is what it is but the person to my right has I don't know how they're going to get in and out so I'm kind of nervous about door dings here but the person behind me has parked way over the line so I can't even back into my spot all the way but that's that so um time to go have a

good day be positive and uh yeah I will uh I'll talk about my caffeine Journey on the way home so stick around for that and I'll see you next time