![\"Basic](\"/491e73fe4836ca5daa93f6aad47f3b4d/basic-architecture.webp\")
Currently, the LoginRadius Identity Platform processes more than 100K requests per second. As our developer and enterprise customers grow, we should be able to scale our APIs to handle staggering loads of requests per second. It won’t be surprising if our APIs have to serve a million requests per second in the future.
\nGenerally, we add more instances, CPU, or memory for supporting high request volumes. However, it has been burdening our infrastructure budget and maintenance.
\nSo, we’re focused on reducing the cost of compute capacity by improving efficiency. In this engineering update, we share our experience with a highly effective, low-risk, large-scale performance and cost optimization with Go.
\nLoginRadius production environment consists of 30+ microservices backed by a cloud-native infrastructure. Most of these services are written in .NET and Node.js. In Q1 2021, we explored the possibilities of improving the performance of our services and optimizing costs.
\nLoginRadius tech stack consists of 30+ microservices. We have different microservices for each business service like Authentication, Web SSO, Federated SSO, etc.
\nCore Applications: Multiple microservices instances for Authentication, Account API, Analytics API, Risk-based Authentication, Backend Jobs, Integrations, and much more,
\nData Storage: MongoDB, Elasticsearch, Redis, and PostgreSQL clusters are used to store a range of datasets for various applications and functionalities.
\nQueues: Amazon Kinesis Data Streams, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), and Azure Queue Storage.
\nOther Services: Services for rate-limiting, bcrypt clusters, feature flags, and more.
\nRouting: AWS load balancers (ALB, NLB, and ELB from AWS) and some nodes running NGINX as a proxy.
\nWe've been utilizing a multi-cloud and multi-region architecture for years based on our and our customers’ data and privacy compliance requirements. However, as our usage (and load) grew with maintenance costs, we relied more on AWS resources.
\nThis is the basic architecture of the environment in the US:
\n
Each request is processed as follows:
\n![\"Request](\"/fc6929115da8e289aaaaa4880d05fe5b/request-processing-flow.webp\")
The above diagram shows two cloud regions — US East and US West.
\nWhen requests come:
\nSo, how do we maintain high availability?
\nEvery availability zone has operating instances of all services (including databases). If one availability zone is down, we still have two availability zones.
\nWe are using a two-pronged scaling approach as scaling based on resource utilization (CPU & Memory) alone is not practical.
\nOnce, our multi-tenant application received a sudden spike in API requests — from thousands of requests to millions. Usually, adding new infrastructure to the existing pool takes around 60 seconds. And application deployment and start-up time were about 90 seconds. Plus, considering other possible delays in scaling, we can lose about 5 minutes for scaling up.
\nFurther, adding the time to gather metrics and make the scaling decision, the total delay can be ~6 minutes. This process can impact all multi-tenant customers' critical applications because user authentication and authorization are essential.
\nWe keep Scaling Buffers to support this kind of scenario.
\n\n\nScaling Buffers: During the Scale-up Time, the Scaling Buffer is the service's highest predicted traffic spike. Depending on the scaling strategy, we store several types of the buffer.
\n
Usually, we set a one million requests concurrency buffer. And based on the traffic pattern, our scaling automation triggers the scaling.
\nOur Platform performance is good. We can quickly handle millions of requests, and our application platform was scalable but not great. Minding the kind of growth we’re expecting, we wanted to improve and prepare our platform for the future.
\nNo. of requests can fluctuate within a few seconds if not minutes or hours, especially with specific events such as holiday shopping, promotional events, Black Friday, etc. So, in the event of traffic spikes, application startup time plays a significant role in scaling.
\nOur application startup time was around 90 seconds, so whenever scaling was happening due to high traffic, scaling took more time. That's why we needed to improve the application startup time.
\nWe used hundreds of AWS c5.2xlarge instances in a production environment to handle daily traffic. During certain events such as holiday shopping and Black Friday, we doubled our infrastructure according to expected traffic with a 20~30% buffer. We kept a high buffer because scaling took time with the existing .NET Core services. As a result, infrastructure cost was burdening as we were inefficiently deploying cloud infrastructure.
\nWe designed our Identity Platform a few years ago. Since then, the technology landscape has dramatically improved and optimized in recent years.
\nAlso, current technology platforms available to build and support our product (Identity Platform) are limited. So, it is crucial to align functionality with proper technology capabilities.
\nWe were facing challenges in customization. It was inflexible and limiting to deliver the required outcomes in more complex situations and use cases.
\nWe did a lot of research to choose the appropriate programming language for our application. We needed a language that would be simple to understand and learn, easy to maintain, and has high concurrency. Our research ended with Go due to its amazing qualities.
\nLet's take a closer look. Why choose Go? What makes Go such an excellent development language? What distinguishes it from the competition?
\nLet's get to the bottom of this by looking at what makes Go special.
\nGo is an open-source programming language developed by Google's Rob Pike, Robert Griesemer, and Ken Thompson. It was initially released in 2009. It has several enhancements and additions based on the C syntax to properly control memory use, manage objects, and enable static or strict typing along with concurrency.
\nConcurrent code can be executed in parallel by separate computer cores or in sequence. Most programming languages lack concurrent execution when working with multiple threads. They often slow down programming, compiling, and execution.
\nGo works with goroutines that are lightweight, low-cost threads. And Go channels allow goroutines to communicate with one another.
\nSequential Processes Communication: They are very similar to threads in Java but lightweight, and the cost of creating them is meager.
\nGo's core value is simplicity: it builds quickly, runs quickly, and is simple to learn. To help speed things up, there are no classes or type inheritance. This makes it easier to build a market-ready product quickly. Furthermore, its simplicity allows for quick and simple maintenance.
\nGo has an excellent standard library that includes a large number of built-in essential type functions and packages that are both practical and straightforward to use: I/O, encoding and decoding, manipulating raw bytes, network utility functions, parsing, debugging, and a lot more are all made simple by particular packages. Testing support is also included in the standard library, so no other dependencies are required.
\nGo has a simpler syntax than other languages and is simple to learn. The syntax of Go is somewhat similar to the C language. Developers can begin with Go in a couple of hours, but they need to spend time understanding best coding practices, especially understanding goroutines for concurrency.
\nWe run benchmarks on our API so that the performance insights help us make decisions about the tech stack. It will paint a clearer picture of where we stand compared to our existing tech stack and what areas need improvement or immediate attention.
\nWe did multiple stress tests on our high usage APIs built separately with .NET Core(Old) and Go (new).
\nWe are using Kubernetes to deploy and manage our microservices and application infrastructure and deploy our application microservices across multiple Pods.
\n| Test Item | \nValue | \n
|---|---|
| Instance Type | \nc5.2xlarge on AWS | \n
| No. of Instances | \n20 | \n
| Reverse Proxy | \nNGINX | \n
| Concurrent Requests | \n20K per second | \n
| Duration | \n5 minutes | \n
| Stress Test Tool | \nGatling | \n
We deployed both applications in the same infrastructure and did the same stress test.\nGolang application has performed much better.
\n![\"Requests](\"/425ce0aea8c01e3281101dd268d0def6/requests-and-responses-per-second-dot-net-core.webp\")
\n ![\"API](\"/d7b965865c218de9b3e71b6559787500/response-time-percentiles-dot-net-core.webp\")
\n ![\"Overall](\"/cd1c01d04ed3965ac3ca5bfce7ba66dc/overall-matrix-dot-net-core.webp\")
\n You can see some errors in Graph - 1(a) after ramping up to 15K users. Also, response time is increasing, as seen in Graph - 1(b).
\nThe application can recover after a couple of seconds, and after that, the application cannot respond. Application scaling is taking time. And existing infra cannot handle this spike, so the application crashes after 18K RPS.
\nWe did the same stress test with the new Go application. It is easily able to handle this kind of concurrency without any issue.
\n![\"Requests](\"/64fd50adc421658ca67fbf096038dc9e/requests-and-responses-per-second-go-lang.webp\")
\n ![\"API](\"/dd50184c7b2f042915816c6374a53590/response-time-percentiles-go-lang.webp\")
\n ![\"Overall](\"/19f6d7a35332ce026578459805a2327d/overall-matrix-go-lang.webp\")
\n The Go application can easily handle 20K ramp-up requests per second without any issue, as shown in Graph -2(a).
\nYou can see in Graph - 2(b) what the response time was after 2 min 47 sec for a couple of seconds. The Go application was scaling during that time. It is scaling quickly, so it is not impacting API request performance, and, also, the application can recover response API time as expected.
\nIn this article, you have learned how we have achieved better API performance by rebuilding them in Go, which were previously built in .NET Core. And you have understood our approach to this transition and how it increased the API performance of the LoginRadius Identity Platform.
\nLeverage LoginRadius Identity Platform to build authentication functionality for your web and mobile apps. Sign up for a free LoginRadius account here
\n","frontmatter":{"title":"Why We Re-engineered LoginRadius APIs with Go?","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"June 06, 2022","updated_date":null,"tags":["Go","API",".NET","LoginRadius"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/da26686e342d94b3007681cd0379367c/58556/high-performance-identity-apis.webp","srcSet":"/static/da26686e342d94b3007681cd0379367c/61e93/high-performance-identity-apis.webp 200w,\n/static/da26686e342d94b3007681cd0379367c/1f5c5/high-performance-identity-apis.webp 400w,\n/static/da26686e342d94b3007681cd0379367c/58556/high-performance-identity-apis.webp 800w,\n/static/da26686e342d94b3007681cd0379367c/99238/high-performance-identity-apis.webp 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/why-we-rebuilt-loginradius-dot-net-core-apis-with-go/"}}},{"node":{"id":"3c5f1273-fe31-522e-acfe-9ba69d4adf7b","html":"A cyber attack is a sequence of actions performed by a threat actor to obtain unauthorized access to a computer, computer network, or other computing systems to intentionally steal data, harm innocent people, or launch attacks from a compromised computer. To launch a cyberattack, cybercriminals utilize many methods, including phishing, ransomware, malware, man-in-the-middle attack, and denial of service, among others.
\nPhishing is used to steal user credentials and sensitive data such as credit card numbers and social security numbers or install malware on a victim's machine. An attacker usually sends fraudulent communications that appear to be from a reputable source. Specifically, scammers send emails or text messages containing malicious links in a manner that seems to originate from legitimate senders.
\nDenial-of-Service (DoS) and Distributed Denial-of-Service (DDoS) attacks send malicious and spam requests to a system or network, severely restricting the ability to perform and serve legitimate users and requests. This attack is frequently used to set up another attack.
\nVishing combines voice and phishing in which an attacker tricks a victim to steal personal and confidential information. It is a social engineering attack as it relies on psychology to convince victims to give sensitive information or perform an action for the attacker's benefit.
\nMalware attacks are common types of cyberattacks in which malware (usually malicious software) performs unauthorized actions like stealing personal, financial, or business information on the victim's system. Malicious software is created in several forms, including ransomware, spyware, and command and control.
\nRansomware is malware that threatens to expose or limit access to data or a computer system by encrypting valuable data or limiting system functionality. Cybercriminals demand monetary incentives (ransom) for releasing the system after encrypting or locking the data. A deadline is typically attached to the ransom demand. If the victim does not pay the ransom on time, the data will be lost permanently, or the ransom will be increased.
\nA man-in-the-middle (MiTM) attack occurs when an attacker intercepts and distributes messages between two participants who think they are interacting directly and securely. Participants who send emails, instant messages, or video conferencing are unaware that an attacker has inserted themselves into the conversation and is collecting and manipulating their information.
\nA Brute-force Attack is an attempt to find a credential, such as a password, using computer-based automated trial and error. The attack involves automated spraying of all possible character combinations and lengths into a password field until a match. Brute-force attacks are successful when the authentication protocol of an online service complements this type of attack. Shared secrets between the service and the user provide the highest probability of success for a brute-force attacker.
\nAny cyberattack that targets an Internet of Things (IoT) device or network is known as an IoT attack. Once the device has been hacked, the hacker can take control of it, steal data, or join a network of infected devices to execute DoS or DDoS attacks.
\nA keylogger is spyware that logs a user's activity by logging keystrokes. Every key pushed on the keyboard is captured and forwarded to a malicious actor when the spyware installs a keylogger on a device. As a result, the attacker will have access to data streams that help find user passwords and other sensitive information. Keylogger spyware is generally installed on the user's device by unintentionally clicking on a malicious link or attachment.
\nCross-site Scripting (XSS) attacks use third-party online resources in which malicious scripts are inserted into a legitimate website or application to obtain a user's information. Attackers commonly employ JavaScript, Microsoft VBScript, ActiveX, and Adobe Flash for XSS attacks. A web app is usually vulnerable to XSS attacks when it receives user input without validating or encoding it in its output.
\nCyberattacks are becoming ever more common and sophisticated, mostly with financial motives.\nWhile preventative cybersecurity tactics vary by attack type, you should follow best security practices and practice IT hygiene for mitigating these attacks.
\nThese cyber-threats are creating more emphasis to stack up security measures. You can follow these Security Tips, which are well-known among LoginRadius’s cybersecurity Experts.
\n","frontmatter":{"title":"Top 10 Cyber Threats in 2022","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"May 31, 2022","updated_date":null,"tags":["Cyber Threats","Security"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/ee3ab87a1a76ecb64198016552132df2/58556/top-10-cyber-threats-in-2022.webp","srcSet":"/static/ee3ab87a1a76ecb64198016552132df2/61e93/top-10-cyber-threats-in-2022.webp 200w,\n/static/ee3ab87a1a76ecb64198016552132df2/1f5c5/top-10-cyber-threats-in-2022.webp 400w,\n/static/ee3ab87a1a76ecb64198016552132df2/58556/top-10-cyber-threats-in-2022.webp 800w,\n/static/ee3ab87a1a76ecb64198016552132df2/99238/top-10-cyber-threats-in-2022.webp 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/top-cyber-threats-in-2022/"}}},{"node":{"id":"2b404803-d316-58d9-a161-7bc978c42d7c","html":"What is a Content Security Policy (CSP), and why is it important?
\nA Content Protection Policy (CSP) is a security standard that adds an extra layer of defense in detecting and mitigating certain kinds of attacks, such as Cross-Site Scripting (XSS), clickjacking, and other code injection threats. CSP is a preventative step against attacks that rely on executing malicious material in a trusted web context, as well as other attempts to bypass the same-origin policy.
\nThe CSP's main purpose is to prevent and report XSS attacks. XSS attacks take advantage of the browser's faith in the server's content. Because the browser trusts the source of the material without additional safety measures, the browser runs all code from a trustworthy origin. It is unable to distinguish which code is legal. Thus any injected malicious code is also executed.
\nThe website administrator can reduce the XSS attack using the CSP by defining trusted source sites for the executable scripts. When we use the CSP header, browsers only allow us to run the script from the whitelisted domains and ignore all other scripts.
\nWe can also use the same-origin policy (SOP) header to prevent the website from accessing data from the other origin. Still, Websites need to include lots of assets from external sources like content delivery networks (CDNs), Google Analytics scripts, fonts, styles, comment modules, social media buttons, etc., so for the modern web, we need to use CSP.
\nOne interesting advantage of a content security policy is we can define the permitted protocols. For example, the sites can restrict browsers from loading content over HTTPS. Some browsers, by default, will not connect to HTTPS but using the content security policy, we can enforce browsers to encrypt conversations with your server.
\nSites may also leverage HTTP Strict-Transport-Security headers to ensure that browsers only connect to the site over encrypted routes.
\nAdding the Content-Security-Policy HTTP header to a web page and setting values for it allows you to restrict what resources the user agent is authorized to load for that page. For example, A page that allows loading external CSS or fonts but not allows loading javascript from the external domains.
\nHTTP response headers are generally used to specify the Content-Security-Policy header, but if needed, you can also use HTML meta tags to provide specific CSP directives at the page level.
\nAn example of adding CSP headers is shown below.
\n Content-Security-Policy: default-src 'self'; img-src *; script-src loginradius.com;An example of adding CSP headers in the HTML tags
\n<meta http-equiv="Content-Security-Policy" content="default-src 'self'">Listed below are a couple of CSP directives and their use cases:
\nDefault-src: This directive serves as a fallback for the other CSP fetch directives. For absent directives like media-src and script-src, the user agent looks for the default-src directive's content and uses it.
Script-src: This directive is used to define locations from which external scripts can be loaded.
Img-src: Specifies sources from which images can be retrieved.
Media-src: This directive is used to define locations from which rich media like video can be retrieved.
Object-src: This directive is used to define locations from which plugins can be retrieved.
Font-src: Specifies permitted sources for loading fonts.
manifest-src: A list of acceptable source locations for web manifests. Web manifests are used by users of Progressive Web Applications to download websites and run them like native mobile apps.
frame-ancestors: A list of acceptable URL locations which this website can load in an iFrame.
form-action: A list of acceptable URL target locations where the website can send form data. It's most likely that you want this value set to self as most websites only submit their form data locally. This property is not covered by default-src above, so make sure you set it.
plugin-types: The list of plugin types that can be loaded from the locations in object-src. Likely that you also want to set this to none.
base-uri: The list of URLs that can be used in HTML base tags on your site.
child-src is used to restrict permitted URLs for JavaScript workers and embedded frame contents, including embedded videos. In Level 3, frame-src and worker-src directives can be used instead to control embedded content and worker processes, respectively.
style-src is used to whitelist CSS stylesheet sources. To allow stylesheets from the current origin only, use style-src 'self'.
connect-src specifies permitted origins for direct JavaScript connections that use EventSource, WebSocket, or XMLHttpRequest objects.
You can find a more updated and complete list maintained by Mozilla here.\nMozilla maintains a more up-to-date and comprehensive list, which can be seen here.
\nAll major modern browsers have supported Content Security Policy.
\nMozilla maintains a more up-to-date and comprehensive list for the CSP support in the browser, which can be seen here.
\nThe default-src directive in the below example policy is set to self. This permits the browser to load resources from the site's origin.
<meta http-equiv="Content-Security-Policy" content="default-src 'self'">The default-src directive in the below example policy permits the browser to load resources from the trusted domain and all its subdomains.
<meta http-equiv="Content-Security-Policy" content="default-src 'self' *.loginradius.com">The above policy permits the browser to load content from loginradius.com as well as any subdomain under loginradius.com.
\nSuppose you are running an e-commerce site and want to ensure that all resources are only loaded via SSL or HTTPS. The below policy ensures that all of the resources on your website load from TLS.
\n <meta http-equiv="Content-Security-Policy" content="default-src https:; script-src https: 'unsafe-inline'; style-src https: 'unsafe-inline'">If you want to allow users of a web application to add images/photos from any source but permits all scripts from trusted sources or specific sources.
\n <meta http-equiv="Content-Security-Policy" content="default-src 'self' img-src *; script-src cdn.loginradius.com;"> The img-src directive allows images to load from anywhere.\nThe script-src directive can only accept executable scripts from cdn.loginradius.com.
If you want to add social widgets like the google+ button, Facebook like, Tweet button on your website, you need to allow external script also like the below policy.
\n<meta http-equiv="Content-Security-Policy" content="default-src 'self' img-src *; script-src cdn.loginradius.com;https://platform.twitter.com; child-src https://plusone.google.com https://facebook.com https://platform.twitter.com;">The Content-Security-Policy-Report-Only Header is a wonderful method to evaluate the effects of a Content-Security-Policy header without really blocking anything on the site. Also, we can get any violation reports using this header. By default, it only sends reports to the developer tools console. If you include a report-to or report-uri directive, it will send a JSON representation of the violation to the provided URI endpoint.
<meta http-equiv="Content-Security-Policy-Report-Only" content="default-src 'self'; report-uri https://report.yourwebsite.com/cspreport;">In the above example, it will not enforce anything. Content-Security-Policy-Report-Only policy only generates reports and sends them to the report URI. The CSP violation report is generated in JSON format.
<meta http-equiv="Content-Security-Policy-Report-Only" content="default-src 'self'; script-src cdn.loginradius.com; report-uri https://report.loginradius.com/cspreport;">In the above example, the policy only allows the script from cdn.loginradius.com.
\n<!DOCTYPE html>\n<html>\n \n<head>\n <title>Content Security Policy</title>\n <script type='text/javascript' src='https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js'></script>\n</head>\n \n<body>\n . . .\n</body>\n \n</html>In the below sample HTML browser trying to download javascript from the other source, but we have allowed javascript src only from the CDN so that the browser will send the following violation report.
\n {\n"csp-report":{\n"document-uri": "https://loginradius.com/test.html",\n"referrer": "",\n"blocked-uri": "https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js",\n"violated-directive": "script-src cdn.loginradius.com",\n"original-policy": "default-src 'self'; script-src cdn.loginradius.com; report-uri https://report.loginradius.com/cspreport;",\n"disposition”: “report"\n}\n}The Content Security Policy will add an additional layer of protection to your web application. CSP is compatible with almost all current browsers and is widely used on the internet as an effective technique for decreasing the threat of cross-site scripting attacks.
\nThe Web Application Security Working Group of the Wide Web Consortium (w3c has already begun work on the specification's next version, Content Security Policy Level 3 and Content Security Policy Level 2 already Candidate Recommendation.
\n","frontmatter":{"title":"Content Security Policy (CSP)","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"July 14, 2021","updated_date":null,"tags":["Secuirty Header","CSP","Content Security Policy"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/a14192a5c3e4cefed740fa3fc0e561e9/58556/content-security-policy.webp","srcSet":"/static/a14192a5c3e4cefed740fa3fc0e561e9/61e93/content-security-policy.webp 200w,\n/static/a14192a5c3e4cefed740fa3fc0e561e9/1f5c5/content-security-policy.webp 400w,\n/static/a14192a5c3e4cefed740fa3fc0e561e9/58556/content-security-policy.webp 800w,\n/static/a14192a5c3e4cefed740fa3fc0e561e9/99238/content-security-policy.webp 1200w,\n/static/a14192a5c3e4cefed740fa3fc0e561e9/7c22d/content-security-policy.webp 1600w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/content-security-policy/"}}},{"node":{"id":"91917bba-2bce-58a1-8ee1-6b0f44a1ea48","html":"There are several Consumer/User data protection regulations in place to protect customer data and privacy. With data protection act like the California Consumer Privacy Act (CCPA) and the European Union's GDPR improving awareness of privacy rights, many data-driven businesses have sought to determine which of their applications contain sensitive data, where the sensitive data goes, and why data is going. As a result, they are re-evaluating their data protection controls for employee and consumer data.
\nYour program must record vast volumes of data to meet business requirements. It's likely that the application log data contains highly sensitive information. Email addresses, URL parameters such as tokens, credit card data, jobs data, Login Credentials, and Official ID Numbers (passport numbers, driver's license numbers, social security numbers), and authentication tokens may be included in specific log messages. It is important to mask sensitive details such as authentication tokens or credit card info while logging.
\nA few years ago, Twitter requested users to reset their passwords. According to Twitter's release, passwords were written in the logs without masking, allowing those with access to the logs to see the password. In most startups or small businesses, all staff members have access to all tools, so masking confidential data is critical. Most log management tools allow you to mask any information until it is saved in the logs. Most of the log monitoring tools provide the feature for masking any information before it will save in the logs.
\nBy default, Nginx logs predefined format is combined. As you know, we can overwrite access log formatting as per the requirement. The NGINX JavaScript plugin can be used to enforce data masking in NGINX logs. This module is a kind of JavaScript implementation for NGINX and NGINX Plus that is intended for server-side use cases and per-request processing. When each request is logged, we run a small amount of JavaScript code to masking the sensitive data.
\nThe NGINX JavaScript module is by default included in the official NGINX Docker image. if your installed version is 1.9.11 or later, then you can install NGINX JavaScript as a prebuilt package for your platform
\nInstall the prebuilt package.
\nUbuntu and Debian systems
\n$ sudo apt-get install nginx-module-njsRedHat, CentOS, and Oracle Linux systems
\n$ sudo yum install nginx-module-njsEnable the module by adding a load module directive in the nginx.conf
\n load_module modules/ngx_http_js_module.so;\n load_module modules/ngx_stream_js_module.so;Reload NGINX service
\n sudo nginx -s reloadAn application can be passed some potentially confidential information as query string parameters like tokens, public key, etc., this information logged as part of the request URI. If your program sends personal information in this manner, you can use the NGINX JavaScript module for masking customer/user personal information in the query string.
\nCreate a new file maskQueryString.js
vi maskQueryString.js\n\n function fnv32a(str) {\n var hval = 2166136261;\n for (var i = 0; i < str.length; ++i ) {\n hval ^= str.charCodeAt(i);\n hval += (hval << 1) + (hval << 4) + (hval << 7) + (hval << 8) + (hval << 24);\n }\n return hval >>> 0;\n }\n\n function maskQueryStringParameters(req) {\n var query_string = req.variables.query_string;\n if (query_string.length) { // Proceed if we have query string\n var keyvaluepairs = query_string.split('&'); // Convert to array of key=value\n for (var i = 0; i < keyvaluepairs.length; i++) { // Iterate through each Key Value pairs pair\n var keyvaluepairs = keyvaluepairs[i].split('='); // Split Key Value pair into new array\n \n if (keyvaluepairs[0] == "token") { // Mask token query paramter\n // Use first 5 digits of masked value\n kvpairs[i] = kvpair[0] + "=" + fnv32a(kvpair[1]).toString().substr(5);\n } \n ekse if (keyvaluepairs[0] == "accountno") { // Mask account no\n // Use first 7 digits of masked value\n keyvaluepairs[i] = keyvaluepairs[0] + "=" + fnv32a(keyvaluepairs[1]).toString().substr(0,7);\n } else if (keyvaluepairs[0] == "email") { // Mask email\n // Use hash as prefix for a single domain\n keyvaluepairs[i] = keyvaluepairs[0] + "=" + fnv32a(keyvaluepairs[1]) + "@sample.com";\n }\n }\n return req.uri + "?" + keyvaluepairs.join('&'); // Construct masked URI\n}\nreturn req.uri; // No query string, return URI\n}The maskQueryStringParameters function loops through and key-value pair in the query string, searching for individual keys that have been identified as containing personal data. The value is converted into a masked value for each of these keys.
\nmaskQueryString.js file in the nginx.config file http {\n\n js_include maskQueryString.js;\n js_set $request_uri_masked maskQueryStringParameters;\n\n log_format custom_log_format '$remote_addr - $remote_user [$time_local] '\n '"$request" $status $body_bytes_sent $request_uri_masked '\n '"$http_host" "$upstream_response_time"'\n '"$http_referer" "$http_user_agent" clientId="$clientid"';\n\n access_log /spool/logs/nginx-access.log custom_log_format;\n....................\n\n } In the above example, I am using the maskQueryStringParameters js function and Since we only need to mask the query string, so using a different variable $requesturimasked and then using the same variable in the log format.
3.Reload NGINX service
\n sudo nginx -s reload Now nginx will store token, accountno, and email in the logs after sanitizing the data.
These best practices will help you keep confidential information out of your logs. It's not a full package that will get you ready for a HIPAA or SOC2 audit, but it'll get you started.
\nThe NGINX JavaScript module provides a quick and efficient approach for adding custom logic like data sanitizing to request processing. I showed how NGINX JavaScript could be used to mask personal data in the log files.
\nYou can get more detail about the Nginx logs and mask from here -\nNginx - Everything you want to know about the Nginx logs in 10 minutes
\n","frontmatter":{"title":"Data Masking In Nginx Logs For User Data Privacy And Compliance","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"May 18, 2021","updated_date":null,"tags":["Nginx","Logs","Data Masking","Compliance","Privacy"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.3333333333333333,"src":"/static/e50f9d5601792a8e4e32b1b89444ca3b/991d2/data-masking-in-nginx-logs-for-user-data-privacy-and-compliance.webp","srcSet":"/static/e50f9d5601792a8e4e32b1b89444ca3b/61e93/data-masking-in-nginx-logs-for-user-data-privacy-and-compliance.webp 200w,\n/static/e50f9d5601792a8e4e32b1b89444ca3b/1f5c5/data-masking-in-nginx-logs-for-user-data-privacy-and-compliance.webp 400w,\n/static/e50f9d5601792a8e4e32b1b89444ca3b/991d2/data-masking-in-nginx-logs-for-user-data-privacy-and-compliance.webp 640w","sizes":"(max-width: 640px) 100vw, 640px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/user-privacy-data-masking-in-nginx-logs/"}}},{"node":{"id":"16e39fb8-be6a-5a88-884e-0f79d1fc4058","html":"Did you know that in only a few minutes, a DDoS attack can bring your website down? Hackers are targeting your site and overloading your network and your server. Your website becomes unresponsive, unavailable and can even go entirely offline. We'll show you how DDoS attacks can be avoided.
\n![\"What](\"/72332960e1243ffa32dc827adaf67f77/What-is-a-DDoS-attack.webp\")
A Distributed Denial of Service (DDoS) is a targeted cyber attack on a website or device where a malicious attacker's flood of traffic is sent from single or multiple sources. The primary purpose of DDoS is to make a machine or network resource unavailable to its genuine users by temporarily or disrupting the services of a host connected to the Internet. If we do not use appropriate security practices or tools, it makes your application into a non-functional situation.
\nThe malicious attacker uses multiple compromised computer systems or devices or IoT devices for the attack. These all compromised devices make the DDoS attacks more effective.
\nDDoS mitigation is a process in which we have used a set of techniques or tools for minimizing or mitigating the impact of distributed denial-of-service (DDoS) attacks on the targeted servers or networks.
\nThe Volume Based Attack DDoS attack is the most common DDoS attack. An attacker uses multiple methods to generate massive traffic volumes to overwhelms a machine's network bandwidth because of creating massive volume traffic that makes it impossible for legitimate traffic to flow into or out of the targeted site. The machine continually has to check the malicious data requests and has no room to accept legitimate traffic. We can detect this type of attack easily.
\nThe Protocol attacks target Layer 3 and Layer 4. Attackers use malicious connection requests for consuming the processing capacity of network infrastructure resources like servers, firewalls, and load balancers.
\nAn SYN flood (half-open attack) is the most common way; in this attack, the attacker sends repeated initial connection requests and overwhelms all available ports on a targeted server machine or device.
\nThe Application attacks target Layer 7, which is the topmost layer in the OSI network model. This layer is closest to the end-user, which means both the OSI application layer and the user interact directly with the software application. These attacks are typically small in volume compared to the other layers attacks, so these attacks are not easy to catch, i.e., a small number of HTTP requests on a login page.
\n![\"How](\"/57071b77d7cf07f146054f334f7b7bb2/How-to-mitigate-DDoS-attack.webp\")
Application traffic monitoring is essential. We can identify most of the attacks using the proper monitoring. Commonly DDoS attacks are made with high volume traffic but DDoS attacks could be possible with a single vulnerable HTTP endpoint.
\nWhenever traffic exceeds a defined threshold, then you should get some alert or notification. The best practice is to have the proper configuration for the alerting in your monitoring tools. It helps you identify the DDoS attack as early as possible and mitigate damage.
\nAccording to organizations size and policies, multiple teams have different responsibilities in infrastructure management. The DDoS attacks happen suddenly and should document the steps that need to follow.
\nDuring the DDoS attack, first, you need to think about minimizing the impact on your application. Team responsibilities for key team members to ensure the organized reaction to the attack should be clear and the first step is defining how it will end.
\nCreate a checklist: Define all the processes and steps in a list, like what tools you will use, who is the right person for contact.
\nCommunication: You should correctly organize all communications and well-defined.
\nResponsibility: Documente all the team member's responsibilities and their reaction.
\nA Web Application Firewall (WAF) is a set of rules or policies that helps protect web applications or APIs from malicious traffic. WAF sits between an application and the HTTP traffic and filters the common web exploits that can affect availability.
\nThere are various WAF solutions available, but you need to analyze which WAF solution is suitable for your application.
\nAttackers can make so many repeated calls on the APIs. It can make resources unavailable to its genuine users. A rate limit is the number of API calls or requests that a user can make within a given time frame. When this limit is exceeded, block API access temporarily and return the 429 (too many requests) HTTP error code.
\nI m adding node js examples to implement the rate limit. multiple npm packages are available for node js
\nNodeJs
\nimport rateLimit from 'express-rate-limit';\n\nexport const apiRatelimit = rateLimit({\n windowMs: 60 * 60 * 1000, // 1 hrs in milliseconds\n max: 100,\n message: 'You have exceeded the 100 requests in 1 hrs limit!', \n headers: true, // it will add X-RateLimit-Limit , X-RateLimit-Remaining and Retry-After Headers in the request \n});\n\n// you can add this in the middleware. it will apply a rate limit for all requests \napp.use(API rate limit);Active cache means if the service first attempts to read from the cache backend and falls back to reading from the actual source. The service is not dependent on requesting the data from the real upstream server. A cache backend is a key-value store (e.g., Redis) or In-Memory cache, and the actual source of data is SQL, MongoDB, etc.
\nPassive cache architecture ensures high volume traffic never hits to actual server or service.
\nI m adding node js examples to implement the passive cache. multiple npm packages are available for node js
\nNodeJs
\nimport nodeCache from "node-cache";\n\nconst myCache = new nodeCache();\nap\n// set object in the cache \n\nobj = { userid: 909887, name: "example" };\n\nsuccess = myCache.set( "userKey", obj, 600 ); // ttl is 600 seconds \n\n\n//read object from the cache \nvalue = myCache.get( "userKey" );\nif ( value == undefined ){\n // handle miss!\n}Several vendors provide DDoS mitigation services as software as a service model. They have charged a license fee the first time then charge according to service usages.
\nThe Cloud-based DDoS mitigation solution has a lot of advantages. They have dedicated staff, better reaction time, High network bandwidth than private networks to perform better in case of volume-based DDoS attack. Multi cross-region availability with auto replica or backup so you can switch to your application load next available region without impacting your actual users, updated policies or rule set, a better experience for handling DDoS attacks.
\nThe DDoS attacks are increasing day by day. Organizations need to prepare for any attack. If the organization does not prepare in advance and any attack happens, that case damage control can take months and impact the organization's reputation.
\nLoginRadius has all processes and policies well defined, 24X7 monitoring by delegating security team. Please see Security Overview document for more information.
\n","frontmatter":{"title":"What is a DDoS Attack and How to Mitigate it","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"February 24, 2021","updated_date":null,"tags":["DDoS","DDos mitigate","DDoS Prevention"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.3333333333333333,"src":"/static/72332960e1243ffa32dc827adaf67f77/58556/What-is-a-DDoS-attack.webp","srcSet":"/static/72332960e1243ffa32dc827adaf67f77/61e93/What-is-a-DDoS-attack.webp 200w,\n/static/72332960e1243ffa32dc827adaf67f77/1f5c5/What-is-a-DDoS-attack.webp 400w,\n/static/72332960e1243ffa32dc827adaf67f77/58556/What-is-a-DDoS-attack.webp 800w,\n/static/72332960e1243ffa32dc827adaf67f77/cc834/What-is-a-DDoS-attack.webp 1024w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/how-to-mitigate-ddos-attack/"}}},{"node":{"id":"319ad830-a6c1-5329-a414-9865884b7c21","html":"Application Security is one of the primary concerns for a software developer. People trust your application and share sensitive or personal information. As a software developer, you need to take care of your application user information security. Authentication and authorization both play critical roles in application security. They confirm the identity of the user and grant access to your website or application.
\nThe process in which confirm the user's identity and provides access to sensitive information is called authentication. Generally, authentication is done through the email/username/password. Authentication using the password is the older and common way, so passwords are a critical component of user's identity security. Password policy is the front line of defense to protect user identity. However, weak passwords may violate compliance standards. A simple or common password could be reversed engineered back to plaintext and sold on the dark web, or result in a costly data breach if compromised.
\nPassword policies and compliance are rules and methods that enforce the user for using a secure and robust password. A billion credentials were stolen last year from multiple data breaches. According to Verizon's Data Breach Report, 81% of data breaches are caused by compromised, weak, and reused passwords. According to National Cyber Security Centre (NCSC) recent analysis, millions of peoples are using easy to guess passwords like 123456. Recently a security researcher claimed he hacked President Trump's tweeter account by guessing his password maga2020! so now we can understand the need for Password Policy & Compliance. You can check the top worst passwords list here.
The Minimum password age policy is to decide how many days minimum users must keep a password before changing it. This password policy.
\nThe \"Enforce password history\" policy is used to make sure the number of unique passwords a user must set before reusing an old password. This is an important policy because password reuse is a common issue – the user feels more comfortable with the old passwords. Using the same password for a long duration for a particular account, it will create a strong chance for the password compromised in some way, such as in a brute force attack. Password age policy shouldn't be efficient until the password history policy. Users must change their password, but they can reuse an old password; the effectiveness of a password age policy is greatly reduced.
\nThe Minimum Password Length policy decides the minimum number of characters needed to create a password. Minimum Password Length should be at least eight characters or more. Longer passwords are generally more secure and harder to crack than short ones. For even greater security, you could set the minimum password length to 14 characters.
\nThe Passwords Complexity Requirements policy make sure user shouldn't use basic passwords. Passwords should be a combination of uppercase, lowercase, and numbers also include some special characters. We can set the following policies in the password Complexity Requirements.
\nThe Password must use following characters combinations
\nThe users shouldn't use the common passwords, so Restrict the use of common passwords. You can refer to this document for a common password list maintained by LoginRadius and this list is dynamic, and it gets updated from time to time.
\nA Password dictionary is a file that contains a list of potential passwords. This feature prevents your user's from setting a password available in the dynamic password dictionary. We are using this dynamic Password Dictionary in the LoginRadius to prevent the use of dictionary passwords.
\nEnabling the Password Audit policy allows you to track all password changes. By monitoring the modifications that are made, it is easier to track potential security problems. This helps to ensure user accountability and provides evidence in the event of a security breach.
\nPassword compliance is a set of rules to enhance user's data security by encouraging users to use strong passwords and use them properly.
\nThe FDA regulates the set of rules for the food, drugs, biologics, medical devices, electronic products, cosmetics, veterinary products, and tobacco products Industries.
\nPasswords for FDA Industry Systems accounts must meet ALL of the following requirements:
\nThe Health Insurance Portability and Accountability Act (HIPAA) enforce a set of rules for sensitive patient data protection. Companies that deal with protected health information (PHI) must ensure HIPAA compliance.
\n!@#$%^&*()_+|~-=\\'{}[]:\";<>?,./;PCI is the set of rules or guidelines for the businesses that are dealing with payment card data.
\nNIST creates a set of rules or guidelines for the businesses that are providing services to the federal government. These guidelines to help federal agencies meet the requirements of the FISMA; however, other organizations reference NIST for strong security standards.
\nI have explained why we needed a strong password policy & compliance. It doesn't matter how strong a password you are using, but bad guys are using new methods or technologies for exposing the user data.\nMost of the data breaches are happing because of Common or weak passwords. MFA, passwordless, or one-time password are providing additional security for a user account.
\nhttps://www.fda.gov/food/online-registration-food-facilities/random-password-generator-fda-industry-systems
\nhttps://uwm.edu/hipaa/security-guidelines/#password
\nhttps://pcipolicyportal.com/blog/pci-compliance-password-requirements-best-practices-know/
\nhttps://spycloud.com/new-nist-guidelines/
\n","frontmatter":{"title":"Password Security Best Practices & Compliance","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"November 12, 2020","updated_date":null,"tags":["Security","Password","Compliance","Passowrd Policy"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/4e0affaac848158a3cb23d1551a0076e/58556/password-security.webp","srcSet":"/static/4e0affaac848158a3cb23d1551a0076e/61e93/password-security.webp 200w,\n/static/4e0affaac848158a3cb23d1551a0076e/1f5c5/password-security.webp 400w,\n/static/4e0affaac848158a3cb23d1551a0076e/58556/password-security.webp 800w,\n/static/4e0affaac848158a3cb23d1551a0076e/99238/password-security.webp 1200w,\n/static/4e0affaac848158a3cb23d1551a0076e/135cd/password-security.webp 1280w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/password-security-best-practices-compliance/"}}},{"node":{"id":"d2908f6b-cd0a-5350-8ca0-0874d84b885b","html":"There are more than 1 million websites or apps are using \"SignIn with Facebook\" or \"Login with Facebook\". Do you know why they are using Facebook login and how it is beneficial in conversion?
\nAccording to multiple surveys, there are more than 3 billion users worldwide are using social media and it is around 50% of the world's population.
\n![\"login-with-facebook\"](\"/6df490f8885582a5f30d79095d46e5de/facebook-login-wow.webp\")
The biggest challenge and most important things are to make user registration and login quick and easy while developing our application. The registration forms required a lot of data that need to be filled by users manually and it causes lost, potential users. Additionally, users need to enter their usernames/emails and passwords in the login forms to authenticate themselves and also need to remember more individual IDs and passwords.
\nSocial Login allows customers to bring their existing social identities and use them to register and log in without creating a new profile explicitly.
\nFacebook is the most favorite social media provider in comparison to other social media providers and the number of active Facebook users growing day by day.
\nIn this article, I will explain how you can implement “Log in with Facebook” on your website or mobile app in a very easy manner.
\nFacebook work on the OAuth 2.0 protocol and most of the social providers like Facebook, Google, Microsoft, Linkedin are supporting OAuth 2.0. Refer to this article Getting Started with OAuth 2.0 to know about OAuth flow.
\nHere you can find the complete step by step guide to create your Facebook Login App.
\nGo to Facebook Developer and log in using your Facebook credentials.
\nNOTE: Please do not log in using a business account as Facebook will not allow you to create an app if you do so.
\n![\"Facebook](\"/e95a7be84484abf90d878b6d4702b8b3/loginadius-facebook-img1.webp\")
Select the My Apps as displayed on the below screen.
\n![\"Facebook](\"/ba396b3223c249de8a434861d9d70b8c/loginadius-facebook-img2.webp\")
Click on Add a New App as displayed in the below screen.
\n![\"Facebook](\"/a135659e8a2ad458e3d40a58a0503c57/loginadius-facebook-img3.webp\")
Input display name and contact email. Once you have done so, click on Create App ID
\n![\"Facebook](\"/76488dc008c92b31560355074b9e8e57/loginadius-facebook-img4.webp\")
In the security check, Complete the security steps and click on the submit button.
\n![\"Facebook](\"/7eee909de66ac55acd1b149241ded179/loginadius-facebook-img5.webp\")
Once you land on the App dashboard, Select Facebook Login, and click Set Up.
\n![\"Facebook](\"/977b5c1dc415d9afff25149ae760706d/loginadius-facebook-img6.webp\")
Click on Settings in the sidebar under Facebook Login. Turn on Client OAuth Login. Turn on Web OAuth Login. Put the valid redirect URL on \"Valid OAuth redirect URIs\". Click on the Save button.
\n![\"Facebook](\"/6a83a418aad05a3d898f7702b0f59d3b/loginadius-facebook-img7.webp\")
Click on Basic under settings in the sidebar. Under App, Domains include your website Url. Enter all the required details. Change the status of the app from Development to Live from the top-right corner. Click on the Save Changes button.
\nNow your Facebook app is ready. You can start implementing Facebook login\n![\"Facebook](\"/870b799fd61f5a1ceb86db30e3a830e7/loginadius-facebook-img8.webp\")
When a user logs into your website or app via Facebook Login, you can access the user's data stored on Facebook. Facebook only allows the basic profile data permissions for a new Facebook app. To get more data according to your business requirement, you need to enable additional permissions on your Facebook app. Facebook is supporting around 42 permissions. You need to choose user data that you want to collect from Facebook.
\nTo grab more than basic profile data points or asking for additional permissions from your users, your Facebook apps go through the review process. Sometimes businesses require some additional permissions on the Facebook app and for that, you need to submit your Facebook app for approval before starting to ask for additional information.
\nThe Facebook app review process is pretty much simple, Please refer to this document here for the Facebook App Review Process.
\nImplement Facebook login to a web page is very easy using Facebook’s JavaScript SDK. Facebook is providing documentation and instructions along with code examples to implement the login system.
\nLet's see how we can add the Facebook login interface. For this, we just need to copy and paste the code provided to us by Facebook. Here I have added some customization to the existing code
\nPlace this code in the body section of your HTML code.
\n<!-- The JS SDK Login Button -->\n\n<fb:login-button scope="public_profile,email" onlogin="checkLoginState();">\n</fb:login-button>\n\n<!-- The JS SDK Login Button -->\n<div id="status">\n</div>Now you need to include the JavaScript section. It'll loads the Facebook SDK JavaScript asynchronously.
\n<!-- Load the JS SDK asynchronously -->\n<script async defer crossorigin="anonymous" src="https://connect.facebook.net/en_US/sdk.js"></script>Next, we'll add the Facebook init function. We just need to replace the app id placeholder with the app id of your app you created in the beginning. You’ll find the placeholder in this line appId: '{your-app-id}'.
\nwindow.fbAsyncInit = function() {\n FB.init({\n appId : '{your-app-id}',\n cookie : true, // Enable cookies to allow the server to access the session.\n xfbml : true, // Parse social plugins on this webpage.\n version : 'v7.0' // Use this Graph API version for this call.\n });Next, we need to add the function that handles the response and alters the page contents based on the type of response. I have added this function at the very top of the scripts section.
\nfunction checkFacebookLoginStatusCallback(response) { // Called with the results from FB.getLoginStatus().\n console.log(' Checking Facebook Login Status');\n console.log(response); // The current login status of the person.\n if (response.status === 'connected') { // Logged into your webpage and Facebook.\n getUserProfile(); \n } else { // Not logged into your webpage or we are unable to tell.\n document.getElementById('status').innerHTML = 'Please login with Facebook into this webpage.';\n }\n }As you can see, the above function receives a response variable and checks it's status. If it is connected it fetches the logged in users info and outputs this information in the console of your browser, that area is where you could build more onto this script to handle the data. When the login is not authorized, this function changed the HTML on your page to ask you to log in.
\nfunction checkLoginState() { // Called when a person is finished with the Login Button.\n FB.getLoginStatus(function(response) { // See the onlogin handler\n checkFacebookLoginStatusCallback(response);\n });\n }\n\n FB.getLoginStatus(function(response) { // Called after the JS SDK has been initialized.\n checkFacebookLoginStatusCallback(response); // Returns the login status.\n });\n\n function getUserProfile() { // Get User Profile using Facebook Graph API after login.\n console.log('Welcome! Fetching your profile information.... ');\n FB.api('/me', function(response) {\n console.log('Successful login for: ' + response.name);\n document.getElementById('status').innerHTML =\n 'Thanks for logging in, ' + response.name + '!';\n });\n }Now if you save this page as login.html and open it in a browser you should see the Facebook Login button. You can also download this code from my GitHub Repo and I have customized it to make it a bit shorter for an explanation.
\nNow just Log In and check the console output. You’ll now see some basic info including id, email, firstname, gender, lastname.
\nThis is all there is to it. Of course, there are many more options and features available in Facebook documentation to discover.
\nSocial Media platforms are growing day-by-day and simplifying the registration and login process. Consumers can use their existing social identities to register and log in. They don't need to create a separate username/password combination so it is simplifying the Sign in/Signup process.
\nLoginRadius supports Sign with Facebook with more than 40 other popular social networks google, twitter, Linkedin, etc. You can implement all the popular social media providers on your website or mobile app in less than 60 seconds.
\nGo announced Go 1.15 version on 11 Aug 2020. Highlighted updates and features include Substantial improvements to the Go linker, Improved allocation for small objects at high core counts, X.509 CommonName deprecation, GOPROXY supports skipping proxies that return errors, New embedded tzdata package, Several Core Library improvements and more.
\nAs Go promise for maintaining backward compatibility. After upgrading to the latest Go 1.15 version, almost all existing Golang applications or programs continue to compile and run as older Golang version.
\nDownload the latest Go Version from Golang.
\nGo 1.14 version announced some changes for the macOS. According to announcement
\n\n\nGo 1.14 is the last release that will run on macOS 10.11 El Capitan. Go 1.15 will require macOS 10.12 Sierra or later.
\n
The lastest macOS 10.15 (Catalina) is no longer supporting 32-bit binaries. Go 1.14 will likely to be the last Go release to support 32-bit binaries on iOS, iPadOS, watchOS, and tvOS (the darwin/arm port). Go continues to support the 64-bit darwin/amd64 port.
\nGo now generates Windows ASLR executables when -buildmode=pie cmd/link flag is provided. Go command uses -buildmode=pie by default on Windows.
The -race and -msan flags now always enable -d=checkptr, which checks uses of unsafe. Pointer. This was previously the case on all OSes except Windows.
Mobile applications were crashing in some devices due to some invalid lld linker versions so now avoiding the crashing the latest Go 1.15 version explicitly selects the lld linker available in recent versions of the NDK. This explicitly lld linker avoids crashes on some devices. it becomes the default NDK linker in a future NDK version.
\nAlways new Go 1.15 brings many minor/major performance improvements. The first major improvement has reduced the executable's binary size by almost 5% - 10% as compared to Go 1.14. As Brad Fitzpatrick tweet, test program down from 8.2MB in Go 1.14 to 3.9MB in 1.15.
\n![\"image](\"/a10871d02098a24f6fe343b6f08f4c46/lr-tweet1.webp\")
![\"image](\"/ca9436ff8af17f9b3eb9d90f38fe1c8a/lr-tweet2.webp\")
Go 1.15 has improved the code build process now unused code is eliminating in the new linker, along with several targeted improvements, such as Clements's CL 230544, which reduces the number of stack and register maps included in the executable.
\nAlways in every new version release Go come with various minor changes and updates to the library. I m including some useful and important changes
\nA new embedded tzdata package was added that permits embedding the timezone database into a program. Importing this package (as import _ \"time/tzdata\") which allows the program to find timezone information without the timezone database on the local system. We can also embed the timezone database by building with -tags timet/zdata. This approach increases the size of the program by about 800 KB. This might be useful and can test some code with the virtualized environments like Go playground.
\nGo older versions were using CommonName field on X.509 certificates as a hostname if there is no Subject\nAlternative Names, would be disabled by default. If Still want to use this legacy behavior then we need to add x509ignoreCN=0 in the GODEBUG environment variable.
\nThe net/url package adds a new URL. The redacted () method that returns the URL as a string. This is proposed in the Issue 34855. It is a very useful improvement for audit logging and security. It's a simple derivation from the URL.String() that masks the password if exists from the string being passed. It does not modify at all to the URL itself but a copy of it.
Go 1.15 has released with various improvements, bug fixes. We can get all improvements, closed proposals and features details for the Go GitHub repository.
\nRESTful programming provides stateless and a uniform interface, Rest API is HTTP-based URLs that hide the back-end infrastructure from the user. Rest APIs provide the back end for modern web and mobile applications.
\nRest APIs are the most important layer in the back-end infrastructure for most modern applications. Cybercriminals are increasingly targeting APIs. Ensuring web API security is the most important and crucial. Let’s see what you can do to ensure REST API security.
\nAPI security start with Http Connection. All requests from clients to your API should be encrypted (HTTPS). Unfortunately, many client HTTP do not enable HTTPS/secure connections by default it’s necessary to enforce that from the server. When Clients who attempt to connect via HTTP should forcefully be redirected to secure HTTPS connections.
\nYou can use HTTP Strict Transport Security security header enforcing Https for web and You can return error for API in case Rest API call on HTTP
\nYou can get a free certificate with Let's Encrypt. SSL provides security from basic API vulnerabilities with almost minimal effort
\nA Distributed Denial of Service (DDoS) is a targeted cyber attack on a web site or device where a malicious attacker flood of traffic is sent from single or multiple sources. the main purpose of DDos is to make a machine or network resource unavailable to its genuine users by temporarily or disrupting services of a host connected to the Internet. if we are not using appropriate security practice or tools then it makes RESTful API into a non-functional situation.
\nAPI DoS attacks are more common these days. Rest APIs utilizations also increasing day-by-day. The organization's dependency is increasing day-by-day because of business needed a unified platform. An attacker can use multiple ways for the DDoS attack so as developer or security engineer you need to implement long-term solution not a temporary
\nAttackers can make so many repeated calls on the APIs. it can make resources unavailable to its genuine users. A rate limit is the number of API calls an app or user can make within a given period. When this limit is exceeded, block API access temporarily and return the 429 (too many requests) HTTP error code.
\nI m adding node js examples to implement the rate limit. multiple npm packages are available for node js
\nNodeJs
\nimport rateLimit from 'express-rate-limit';\n\nexport const apiRatelimit = rateLimit({\n windowMs: 60 * 60 * 1000, // 1 hrs in milliseconds\n max: 100,\n message: 'You have exceeded the 100 requests in 1 hrs limit!', \n headers: true, // it will add X-RateLimit-Limit , X-RateLimit-Remaining and Retry-After Headers in the request \n});\n\n// you can add this in the middleware. it will apply rate limit for the all requests \napp.use(apiRatelimit);Active cache means if the service first attempts to read from the cache backend and falls back to reading from the actual source. The service is not dependent or requesting the data from the actual upstream server. a cache backend is a key-value store (e.g. Redis) or In-Memory cache and the actual source of data is an SQL, MongoDB, etc.
\nPassive cache architecture ensures high volume traffic never hit to actual server or service.
\nI m adding node js examples to implement the passive cache. multiple npm packages are available for node js
\nNodeJs
\nimport nodeCache from "node-cache";\n\nconst myCache = new nodeCache();\n\n// set object in the cache \n\nobj = { userid: 909887, name: "example" };\n\nsuccess = myCache.set( "userKey", obj, 600 ); // ttl is 600 seconds \n\n\n//read object from the cache \nvalue = myCache.get( "userKey" );\nif ( value == undefined ){\n\t// handle miss!\n}Sensitive data exposure happens when an application, organization, or other entity unable to properly secure sensitive data. It is different from a data breach, it includes personal information, tokens, etc. We can make sure sensitive data security using
\nmultiple ways which include encryption at rest or in transit and masking
Cross-Site Scripting (XSS) attacks are a type of injection, in which attacker aims to execute malicious scripts in a web browser of the victim. an attacker can transfer untrusted data into the API as part of a query or command.which can result in an attacker obtaining unauthorized access to information or carry out other damages.
\nAt the point where user input is received, filter as strictly as possible based on what is expected or valid input.
\nTo prevent XSS in HTTP responses that aren't intended to contain any HTML or JavaScript, you can use the Content-Type and X-Content-Type-Options headers to ensure that browsers interpret the responses in the way you intend.
\nIf you want to know more details about the security headers. Please go to Security Headers
\nAs a last line of defense against attackers, you can use Content Security Policy (CSP) to reduce the severity of any XSS vulnerabilities that still occur.
\nNode js we can use xss-clean package. This dependency will prevent users from inserting HTML & Scripts on input.
NodeJs
\nimport xssClean from "xss-clean";\n\n// Use this as middleare \napp.use(xssClean())We can discover suspicious activity using proper logging and monitoring. When We have insufficient logging and monitoring in that case sometimes we can miss some system access or user activity logs, a step of the particular activity and security alerts.
\nA lot of logging and monitoring tools are available. We can choose the best tools as per our requirement also we can define some policies like data retention policy that includes how far backlogs will be kept. Instrument your API access actions to record key metrics and events. Keep logs indexable and searchable.
\nEach API should limit access, API only able to perform what tasks they need to do. We can do this with Role-Based Access, separate read/write API Keys, OAuth Scopes, and permissions systems. This minimizes the chances that you’ll accidentally expose a sensitive field.
\nSometimes, people find security vulnerabilities, and they would like to report them so the vendor or the developer can fix them. you must have a public contact point where security issues can be reported.
\nWe can create a security.txt file on the site. security.txt is a proposed standard for websites' security information that will allow security researchers to easily report security vulnerabilities. The \"security.txt\" that is similar to robots.txt. security.txt files have been adopted by Google, GitHub, LinkedIn, and Facebook
You can easily create secuirty.txt file using the securitytxt.org
Now days lot of data breaches are happing. We can save mostly data breaches after following some basic security guidelines.You have to pay attention to security during Rest API development. I have covered most of the general Rest API security issues with resolution. these guidelines will help you for developing more secure and quality REST API service.
\n","frontmatter":{"title":"Best Practice Guide For Rest API Security | LoginRadius","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"August 20, 2020","updated_date":null,"tags":["RestAPI","Rest API","Rest API Security","Best Practice","Rest API Developer Guide","Security"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7699115044247788,"src":"/static/0072087246a78a28a06a6c6744a6823a/58556/index.webp","srcSet":"/static/0072087246a78a28a06a6c6744a6823a/61e93/index.webp 200w,\n/static/0072087246a78a28a06a6c6744a6823a/1f5c5/index.webp 400w,\n/static/0072087246a78a28a06a6c6744a6823a/58556/index.webp 800w,\n/static/0072087246a78a28a06a6c6744a6823a/99238/index.webp 1200w,\n/static/0072087246a78a28a06a6c6744a6823a/7c22d/index.webp 1600w,\n/static/0072087246a78a28a06a6c6744a6823a/25f09/index.webp 1920w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/best-practice-guide-for-rest-api-security/"}}},{"node":{"id":"ceb2a4e9-a19e-53ca-9623-5425dfcaeab2","html":"A hosts file which is used by operating systems to map a connection between an IP address and domain names before going to domain name servers. This file is a simple text file with the mapping of IPs and domain names.
\nYou can use this to block advertisers, trackers, block marketing, or third party websites, block ads, banners, 3rd party page counters, or sites to protect your privacy.
\nIf you don’t want your children to open some websites you can block them by the hosts file. You can decide entirely what you wish to block and even most hijackers and possibly unwanted programs.
\nWe can utilize it as a firewall in our local system. The hosts file is to block Spyware and/or ad Networks you can add all the Spyware sites & ad Networks domain names in your hosts file also you can block dangerous sites, ransomware sites, blockchain sites.
\nYou know during the development, you need to run your web application on the localhost for verification. Websites can run on the localhost, 127.0.0.1, or localhost IP address. There are some limitations in the localhost, you want to review your website on the custom domain before launching on the public domain suppose you have developed the e-commerce. You want to debug some payment gateway issue but your payment gateway is not supporting localhost URL or IP in the case of successful payment. You can add custom domain in the hosts file and validate the payment process on localhost
\nWindows operating system we need to open the host file
\n ![\"Start](\"/20915351d9e41e6e17b66f533f7db315/windows1.webp\")
Open the hosts file. Click on File > Open and Copy and Paste the following path`
\nc:\\Windows\\System32\\Drivers\\etc\\hosts
![\"edit-hosts-file-windows\"](\"/a52d88025ba356b78b68017472da1df7/windows2.webp\")
You can edit the hosts file
\nSuppose you want to block facebook.com on your system and want to add a custom domain for your website. Just Copy and Paste following Lines
\n0.0.0.0 www.facebook.com\n127.0.0.1 www.customdomain.com ![\"Save-hosts-file-windows\"](\"/3074615f157eeced9b2f4894885241dd/windows3.webp\")
After finishing the Editing, Save your hosts file
\n ![\"Save-hosts-file-windows\"](\"/ead0cbcf828255f6f85c0fe51ebabff0/windows4.webp\")
Open your browser and try to access www.facebook.com and see you can’t access this site
\n ![\"Facebook-windows\"](\"/220ec47e648a76dc4f5fb2e09725d9cf/windows5.webp\")
Use following instructions for Linux
\nIn the Linux terminal window, open hosts file using a favorite text editor
\n$ sudo vim /etc/hosts ![\"linux-hosts-file\"](\"/3095554fa34ec943f59ec582c335a595/linux1.webp\")
It will prompt for the password, enter your administrator password.
\n ![\"linux-hosts-file-edit\"](\"/0a619c3369d143ad0fdc5e2c64eb80e0/linux2.webp\")
Use following instructions for macOS
\n ![\"linux-hosts-file-edit\"](\"/e0b8a6efe1386a10aa3419333583826c/mac1.webp\")
sudo vim /etc/hosts in the terminal\n-- It will prompt for the password, enter your administrator password\n-- Enter administrator Password\n ![\"mac-password\"](\"/f6ba574fac85962782fddd1d5ea01009/mac2.webp\")
Using the vim text editor you can easily edit you. The macOS hosts file is also similar to the windows and Linux hosts file. I am blocking Facebook and adding custom domain here as well.
\n ![\"mac-hosts-file-edit\"](\"/ca8db5b6c7015e56d24e0eab54598b77/mac3.webp\")
![\"edit-hosts-file-mac\"](\"/220ec47e648a76dc4f5fb2e09725d9cf/mac4.webp\")
The hosts file is found on all operating systems. The hosts file is a powerful tool. It can make your computer more secure and safer by blocking malicious sites
\n","frontmatter":{"title":"Benefits and usages of Hosts File","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"July 20, 2020","updated_date":null,"tags":["Computer tricks","Networking","Hosts File"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.4184397163120568,"src":"/static/54b234435c9e01dc08a3d668cceb489e/991d2/index.webp","srcSet":"/static/54b234435c9e01dc08a3d668cceb489e/61e93/index.webp 200w,\n/static/54b234435c9e01dc08a3d668cceb489e/1f5c5/index.webp 400w,\n/static/54b234435c9e01dc08a3d668cceb489e/991d2/index.webp 640w","sizes":"(max-width: 640px) 100vw, 640px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/hosts-file/"}}},{"node":{"id":"0ad9f87c-7360-51f9-ac8a-16c533d7b145","html":"When we visit any website in the browser, the browser sends some request headers to the server and the server responds with HTTP response headers. These headers are used by the client and server to share information as a part of the HTTP protocol. Browsers have defined behavior of the web page according to these headers during communication with the server. These headers are mainly a combination of key-value pairs separated by a colon :. There are many HTTP headers, but here I'm covering some very useful web security headers, which will improve your website security.
As you know, nowadays too many data breaches are happening, many websites are hacked due to misconfiguration or lack of protection. These security headers will protect your website from some common attacks like XSS, code injection, clickjacking, etc. Additionally these headers increases your website SEO score.
\nHTTP Strict Transport Security security header helps to protect websites against man-in-the-middle attacks and cookie hijacking. Let's say, you have one website www.example.com and you have purchased SSL certification for migrating your website from HTTP to HTTPS. Now suppose old users are still opening your website using the HTTP, so they may redirect your HTTP users to HTTPS via redirection as a solution. Since visitors may first communicate with the non-encrypted version of the site before being redirected. This creates an opportunity for a man-in-the-middle attack. The HTTP Strict Transport Security header informs the browser that it should never load a site using HTTP and should automatically convert all attempts to access the site using HTTP to HTTPS requests.
Strict-Transport-Security: max-age=<expire-time>\n Strict-Transport-Security: max-age=<expire-time>; includeSubDomains\n Strict-Transport-Security: max-age=<expire-time>; preload| Directives | \nExplanation | \n
|---|---|
max-age=<expire-time> | \nThis directive allows us to specify the amount of time (in seconds) that the content of the header will be stored in the browser cache. | \n
includeSubDomains | \nIf this optional parameter is specified, this rule applies to all of the site's subdomains as well. | \n
Preload | \nThis is not part of the specifications. Please see the Preloading Strict Transport Security | \n
X-XSS header helps protect websites against script injection attacks. When an attacker injects malicious JavaScript code into an HTTP request for accessing confidential information such as session cookies, at that time HTTP X-XSS-Protection header can stop the browsers from loading suce web pages, whenever any detect is reflected cross-site scripting (XSS) attacks. XSS is a very common and effective attack.
\n X-XSS-Protection: 0 \n X-XSS-Protection: 1 \n X-XSS-Protection: 1; mode=block \n X-XSS-Protection: 1; report=<reporting-uri>| Attribute | \nDescription | \n\n | \n | \n |
|---|---|---|---|---|
0 | \nDisable the XSS Filtering | \n\n | \n | \n |
1 | \nEnable the XSS Filtering and remove the unsafe part | \n\n | \n | \n |
1; mode=block | \nBrowser prevents the entire page from rendering when an XSS attack is detected. | \n\n | \n | \n |
1; report=<reporting-URI> (Chromium only) | \nWhen a cross-site scripting attack is detected, the browser will remove the unsafe parts from the page and report the violation on the reporting URL. | \n\n | \n | \n |
The X-Frame-Options HTTP response header can be used to instruct the browser whether a web page should be allowed to render a <frame>, <iframe>, <embed> or <object> element on website or not.
This header protects users against ClickJacking attacks. An attacker uses multiple tricks to trick the user into clicking something different than what they think they’re clicking.
\n X-Frame-Options: DENY \n X-Frame-Options: SAMEORIGIN| Directives | \nExplanation | \n
|---|---|
DENY | \nThis will not allow to page rendering in the <frame>, <iframe>, <embed> or <object> | \n
SAMEORIGIN | \nThis will allow the page to only be displayed in a <frame>, <iframe>, <embed> or <object> on the same origin. | \n
X-Content-Type-Options response header prevents the browser from MIME-sniffing a response away from the declared content-type. A MIME-sniffing vulnerability allows an attacker to inject a malicious resource, such as a malicious executable script, Suppose an attacker changes the response for an innocent resource, such as an image. With MIME sniffing, the browser will ignore the declared image content type, and instead of rendering an image will execute the malicious script.
\nSyntax
\n X-Content-Type-Options: nosniff nosniff - Blocks a request if the request destination is of type:
Content-Security-Policy header is used to instruct the browser to load only the allowed content defined in the policy. This uses the whitelisting approach which tells the browser from where to load the images, scripts, CSS, applets, etc. If implemented properly, this policy prevents the exploitation of Cross-Site Scripting (XSS), ClickJacking, and HTML injection attacks.
\n Content-Security-Policy: <policy-directive>; <policy-directive>More information about the Content Security policy can be found on Mozilla’s website .
\nAs a web developer, you can follow this guide for adding security headers to your website. HTTP headers can be configured on the server to enhance the overall security of the web application. You can easily validate your website security headers. Multiple free online tools are available to check the same:
\nThese headers reduce the potential vulnerabilities of the application. Alongside these headers will add one layer of security still we need to add more layer's of security in our web application
\n","frontmatter":{"title":"HTTP Security Headers","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"July 15, 2020","updated_date":null,"tags":["Security","HTTP Headers"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/4bea535329516b2e7baca9bfc373ccfc/991d2/http-security-headers.webp","srcSet":"/static/4bea535329516b2e7baca9bfc373ccfc/61e93/http-security-headers.webp 200w,\n/static/4bea535329516b2e7baca9bfc373ccfc/1f5c5/http-security-headers.webp 400w,\n/static/4bea535329516b2e7baca9bfc373ccfc/991d2/http-security-headers.webp 640w","sizes":"(max-width: 640px) 100vw, 640px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/http-security-headers/"}}},{"node":{"id":"cbbb8461-7937-51de-9ae7-4aae13825d49","html":"Cloud Cost Optimization is the method of reducing the total cloud spending by finding mismanaged capital and scale-up of Right Sizing computing services. CCO only charge for the services you use, the cloud gives companies infinite scalability and lower IT costs.
\nOn-Premise Infrastructure was only option around 10 years ago but managing On-premise Infrastructure include the extra cost with other challenges, Challenges are Network, Hardware, Cooling, Power, Space also needed technical staff for taking care all infrastructure
\nThe Colocation is when a business places its own server in a third-party data center and uses its infrastructure services. Colocation takes responsibility for Network, Hardware, Cooling, Space also business not need to worry about technical man force for managing servers
\nStill business have more challenges OS software, Application security, Data storage
\nThe Cloud service provider is providing solutions for all the challenges. The cloud service provider takes total responsibility for hardware, network, space, power, maintaining, and security also they are managing all technical workforce for the whole infrastructure
\n ![\"infra](\"/9475f7e05b9e351f919a2b693d2812da/infra1.webp\")
Reducing Cloud cost is not a one time task. it is a recurring process. We need to identify underutilized resources, Right size machine, reserving capacity for higher discounts also need to optimization Application for reducing hardware cost. Let's start
\nCloud infrastructure is increasing day by day. Choosing the right cloud provider is the most important decision for long term success. The big three cloud providers are Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). These three cloud providers are providing mostly services so they might be confused about choosing the right cloud provider. We can easily evaluate the right cloud provider from the following steps
\nAll Big Cloud Providers are providing discounted Instances or spot instances. Spot instances are unused instances. Cloud Providers offered up to 90% discount on these instances compared to on-demand or reserved instances. The majority of organizations have some workloads that are not critical, We can reduce the cost for not critical workload by using spot instances. AWS, Azure, and Google (GCP) all provide the option to use Spot Instances.
\nIdentify the right size for the Instance, not an easy task. We need to configure multiple matrices in our cloud service provider. Key metrics to look for are CPU and memory usage. Identify instances with a maximum CPU usage and memory usage of the month.
\nGenerally, people make mistakes during checking metrics. Most of the people prefer averages in monitoring but averages mislead the measurement. Let understand this by example
\nYou are running an online technology tutorial site. Suppose 1million active users on your website in normal days. Now you are hosting weekly technology webinars in your platform. During webinars time your platform traffic should increase. At that time CPU usages and memory usages should be high compared to the rest of the time. Now suppose you have checked the 24 hours average CPU usages and memory usages. It was around 40% and you have decided this is underutilized and scale down the lower size instance. It can impact you 5% - 10% traffic Let’s look at a real-world example. This chart shows the overall CPU usages
\n ![\"image](\"/205f2fae49dd4ac8281b545c5341d6ce/image1.webp\")
First Image showing Average CPU utilization. You can see Maximum CPU was 18.6 %
\nNow, let’s check the CPU 99th percentile:\n![\"image](\"/6c898bdf0b5b329e83f2c089d0b71b28/image2.webp\")
As expected, the 99th percentile is higher than the average. 99th percentile is around 93%
\nThe conclusion is We need to always check the 99th percentile. The average can mislead and it can impact your users.
\nWe can create different types of resources and applications utilization monitoring matrix and based on the data we can configure multiple alarms. You can configure notification or alarms on email, SMS from the Cloud provider dashboard. you can also configure alarms that automatically stop or terminate EC2 instances or VM when instance unused or underutilized according to the configured threshold. During the development or some POC as a developer or devops person we have to create some instances or resources but sometimes we forgot to terminate the instances or resources. You can minimize the this extra cost by creating a group of alarms that sends an email notification to developers whose instances have been underutilized or ideal for some hours, then terminates an instance. It will save the overall infra cost. different cloud providers provide different ways for creating these type alarms
\nYou can save around 75% cost for your Non - Production Development, Staging, and QA environment. Non - Production environment generally needed during working days. You can turn off these servers during off-hours. Cost-saving depends on the infrastructure size, it can be hundred, thousands of dollars
\nYou can create automation scripts for infrastructure deployment. Schedule the script in your cloud provider
\nApplication in-memory cache reduces the cost of transferring data in the network and overall application performance because reduce the traffic between the database servers or any other external application reduce the network level cost in the cloud also caching improves the efficiency and accessibility of data that used repeatedly or frequently accessed. Suppose your application is fetching user configuration or settings in every request from the database server. You can keep this type of configuration, which is not changing frequently in the in-memory cache. it will save a lot network-level cost
\nData Transfer cost mostly hidden or Some time we don't take care of it. Generally, data transfer is free in the same region between different services Storage, Compute service, etc.\nIf you do a lot of cross-region transfer, it will increase your network data transfer cost also if you will deploy multiple services in the same region it will improve application performance
\nThis includes knowing what you spend in detail, how specific services are billed, and the ability to display how (or why) you spent a specific amount Here, keep in mind key capabilities such as the ability to create shared accountability, hold frequent cost reviews, analyze trends, and visualize the impact of your actions on a near-real-time basis. You can also use cost controls like budget alerts and quotas to keep your costs in check over time.
\nEnterprises or Mid Level companies are adopting multi-cloud infrastructure. Consider this recent prediction from IDC: “By 2020, over 90% of enterprises will use multiple cloud services and platforms.” Or this one from 451 Research: “The future of IT is multi-cloud and hybrid with 69% of respondents planning to have some type of multi-cloud environment by 2019.”
\nOrganizations need to develop a cost optimization culture and awareness. Cost optimization is an ongoing activity in the organization. Need to decide someone responsible for the cost optimization it can be an Engineering team or DevOps team. Most cloud providers provide billing alarm’s they can alert you in case of cost increment also we can configure budget in the Cloud provider dashboard.
\n","frontmatter":{"title":"Cloud Cost Optimization in 2021","author":{"id":"Vijay Singh Shekhawat","github":"code-vj","avatar":null},"date":"July 09, 2020","updated_date":null,"tags":["Cloud","Optmization"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.408450704225352,"src":"/static/9b95fca99dc1c53ce661e89fd0341f6d/58556/cloud.webp","srcSet":"/static/9b95fca99dc1c53ce661e89fd0341f6d/61e93/cloud.webp 200w,\n/static/9b95fca99dc1c53ce661e89fd0341f6d/1f5c5/cloud.webp 400w,\n/static/9b95fca99dc1c53ce661e89fd0341f6d/58556/cloud.webp 800w,\n/static/9b95fca99dc1c53ce661e89fd0341f6d/99238/cloud.webp 1200w,\n/static/9b95fca99dc1c53ce661e89fd0341f6d/7c22d/cloud.webp 1600w,\n/static/9b95fca99dc1c53ce661e89fd0341f6d/8dda0/cloud.webp 1747w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Vijay Singh Shekhawat","slug":"/engineering/cloud-cost-optimization/"}}}]},"authorYaml":{"id":"Vijay Singh Shekhawat","bio":"He is the Lead Product Architect @LoginRadius. He loves working with technology and building something new. He is also a breakthrough thinker, DevOps guy, and cybersecurity enthusiast.","github":"code-vj","stackoverflow":"1063165","linkedin":"vijaydotnetdeveloper","medium":null,"twitter":null,"avatar":null}},"pageContext":{"id":"Vijay Singh Shekhawat","__params":{"id":"vijay-singh-shekhawat"}}},"staticQueryHashes":["1171199041","1384082988","2100481360","23180105","528864852"]}