The Internet and .txt files in the era of AI, ML and data scrapping

For the non developers, the web industry is using .txt files at the root of a domain to provide guidance for different kind of bots wandering on the Internet and indexing/scrapping data. The bots can ignore the instructions, declare a different User-Agent or other kind of evasion methods. robots.txt - it instructs crawlers on which […]

Read More
90% decrease in transactional emails cost – going to AWS SES

Whan I am developing something I am more focused on the engineering behind and not on setting up basic things. I just use any available system and I will deal with the optimization at a later stage. This was the case also for rotrafic.xyz, it uses a double opt-in for registration so I used Sendingblue […]

Read More
About server monitoring (with Netdata) as a way to detect malicious activities, a real case of a compromised domain and decoding backdoor payloads

The backdoor was planted in /wp-includes/ and the filename started with a dot in an attempt to hide the file .query.php

Read More
Follow-up Facebook Shops and some extra thoughts about social media

Today my Facebook news-feed was like in the picture below, what I wrote more than a year ago actually happened My opinion about Facebook Shops: Facebook Shops will only dilute even more the news feed. This reveals 2 issues, on both sides of the system: Facebook still does whatever it wants to do. As there is no […]

Read More
How much time does it take the update of statistics from Search Console?

Almost one month. I did an experiment. Having some pages on a well indexed domain with multiple daily visits from GoogleBot. Initially with a red CLS so the pages were "poor URL", let it stabilize, then modified the pages to orange CLS so the pages went to "URLs need improvement" Update done on 27.04.2021 was visible in Search Console on 17-18.05.2021. I […]

Read More