Difference between revisions of "UBIFS"

From eLinux.org
Jump to: navigation, search
(Created page with "== Introduction == [http://www.linux-mtd.infradead.org/doc/ubifs.html UBIFS] is a filesystem that works on top of [http://elinux.org/File_Systems#UBI UBI] volumes * UBIFS pres...")
 
(Added compression type to mkfs.ubifs options)
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Introduction ==
 
== Introduction ==
 +
UBIFS is a filesystem that works on top of [http://elinux.org/File_Systems#UBI UBI] volumes
  
[http://www.linux-mtd.infradead.org/doc/ubifs.html UBIFS] is a filesystem that works on top of [http://elinux.org/File_Systems#UBI UBI] volumes
+
* the UBIFS home page is at: http://www.linux-mtd.infradead.org/doc/ubifs.html  
 
+
 
* UBIFS presentation slides: [http://www.linux-mtd.infradead.org/doc/ubifs.odp ubifs.odp]
 
* UBIFS presentation slides: [http://www.linux-mtd.infradead.org/doc/ubifs.odp ubifs.odp]
 +
 +
It represents a next-generation flash filesystem, compared to JFFS2, and (via UBI) operates
 +
on raw flash rather than on block-based media (such as eMMC or traditional hard drives).
 +
 +
=== History ===
 +
UBIFS was written by developers at Nokia with help from the University of Szeged.  Included in these
 +
developers was the current (as of 2012) maintainer of UBIFS, Artem Bityuski.
 +
The filesystem was mainlined in the Linux kernel in July, 2008, in kernel version 2.6.27.
 +
 +
=== Features ===
 +
The [http://www.linux-mtd.infradead.org/doc/ubifs.html UBIFS home page] has more detailed information,
 +
but in summary, UBI has the following features relative to other flash filesystems:
 +
* UBI/UBIFS scales to large flash sizes better than JFFS2
 +
* Good fault tolerance, via a number of features
 +
* Built-in on-the-fly compression
 +
* Good runtime performance
 +
 +
=== Fastmap support ===
 +
As of kernel version 3.7, UBI fastmap support was added to the kernel, to overcome a scalability
 +
issue with the time required to mount UBIFS on large flash media.
 +
 +
I specifically deals with the scalability issue described here:
 +
http://www.linux-mtd.infradead.org/doc/ubifs.html#L_scalability
 +
 +
 +
A description of the feature is at: http://lwn.net/Articles/517422/
 +
 +
"UBI Fastmap is an optional feature which stores the physical to
 +
logical eraseblock relations in a checkpoint (called fastmap) to reduce
 +
the initialization time of UBI. The current init time of UBI is
 +
proportional to the number of physical erase blocks on the FLASH
 +
device. With fastmap enabled the scan time is limited to a fixed
 +
number of blocks."
  
 
== UBIFS vs. YAFFS2 comparisons for 2.6.31.1 ==
 
== UBIFS vs. YAFFS2 comparisons for 2.6.31.1 ==
  
First, see our [[Flash_Filesystem_Benchmarks]] for more recent benchmarks.
+
Note: see our [[Flash_Filesystem_Benchmarks]] for more recent benchmarks.
  
 
Hardware: MIPS, 403MHz CPU, 1GB Nand Flash
 
Hardware: MIPS, 403MHz CPU, 1GB Nand Flash
Line 107: Line 140:
 
* -e - Logical Erase Block (LEB) size.
 
* -e - Logical Erase Block (LEB) size.
 
* -c - Max LEB count. (vol_size/LEB)
 
* -c - Max LEB count. (vol_size/LEB)
 +
* -x - Compression type: lzo (default), favor_lzo, zlib, none
 
* -r - Path to root filesystem.
 
* -r - Path to root filesystem.
 
* ubifs.img - Temporary image file.
 
* ubifs.img - Temporary image file.
Line 117: Line 151:
 
* -O - VID header offset from start of PEB.
 
* -O - VID header offset from start of PEB.
 
* ubi.ini - UBI image configuration file.
 
* ubi.ini - UBI image configuration file.
 +
 +
== Mounting UBI Image on PC using nandsim ==
 +
 +
First create a simulated NAND device (this one is 256MB, 2048 page size).
 +
<number>_id_byte= corresponds to the ID bytes sent back from the NAND.
 +
 +
$ sudo modprobe nandsim first_id_byte=0x20 second_id_byte=0xaa third_id_byte=0x00 fourth_id_byte=0x15
 +
 +
Check it was created.
 +
$ cat /proc/mtd
 +
dev:    size  erasesize  name
 +
mtd0: 10000000 00020000 "NAND simulator partition 0"
 +
 +
Next, attach it to a mtd device.
 +
$ sudo modprobe ubi mtd=0
 +
 +
I had to detach it prior to formatting it.
 +
$ sudo ubidetach /dev/ubi_ctrl -m 0
 +
 +
If that ubidetach step fails when you enter it, just proceed to the next step to format the mtd device.
 +
$ sudo ubiformat /dev/mtd0 -f <image>.ubi
 +
ubiformat: mtd0 (nand), size 268435456 bytes (256.0 MiB), 2048 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
 +
libscan: scanning eraseblock 2047 -- 100 % complete 
 +
ubiformat: 2048 eraseblocks have valid erase counter, mean value is 1
 +
ubiformat: flashing eraseblock 455 -- 100 % complete 
 +
ubiformat: formatting eraseblock 2047 -- 100 % complete 
 +
 +
Then, attach it.
 +
$ sudo ubiattach /dev/ubi_ctrl -m 0
 +
UBI device number 0, total 2048 LEBs (264241152 bytes, 252.0 MiB), available 0 LEBs (0 bytes), LEB size 129024 bytes (126.0 KiB)
 +
 +
Make a target directory, and mount the device.
 +
$ mkdir temp
 +
$ sudo mount -t ubifs ubi0 temp

Latest revision as of 07:32, 22 October 2012

Introduction

UBIFS is a filesystem that works on top of UBI volumes

It represents a next-generation flash filesystem, compared to JFFS2, and (via UBI) operates on raw flash rather than on block-based media (such as eMMC or traditional hard drives).

History

UBIFS was written by developers at Nokia with help from the University of Szeged. Included in these developers was the current (as of 2012) maintainer of UBIFS, Artem Bityuski. The filesystem was mainlined in the Linux kernel in July, 2008, in kernel version 2.6.27.

Features

The UBIFS home page has more detailed information, but in summary, UBI has the following features relative to other flash filesystems:

  • UBI/UBIFS scales to large flash sizes better than JFFS2
  • Good fault tolerance, via a number of features
  • Built-in on-the-fly compression
  • Good runtime performance

Fastmap support

As of kernel version 3.7, UBI fastmap support was added to the kernel, to overcome a scalability issue with the time required to mount UBIFS on large flash media.

I specifically deals with the scalability issue described here: http://www.linux-mtd.infradead.org/doc/ubifs.html#L_scalability


A description of the feature is at: http://lwn.net/Articles/517422/

"UBI Fastmap is an optional feature which stores the physical to
logical eraseblock relations in a checkpoint (called fastmap) to reduce
the initialization time of UBI. The current init time of UBI is
proportional to the number of physical erase blocks on the FLASH
device. With fastmap enabled the scan time is limited to a fixed
number of blocks."

UBIFS vs. YAFFS2 comparisons for 2.6.31.1

Note: see our Flash_Filesystem_Benchmarks for more recent benchmarks.

Hardware: MIPS, 403MHz CPU, 1GB Nand Flash

IOZone results: 4M, 8M & 16M file sizes in 980MB partition.

  • mount time
    • "1st mount" : time for mounting just after "flash_eraseall".
    • "Empty" : time for mounting after "1st mount".(there's no files in partition)
    • "Full" : time for mounting after creating files util the partition is full.(file size is random.)
    • "Ubiattach" time for attaching 980MB partition into the ubi layer using ubiattach util.
Ubiattach Ubifs Yaffs2
1st mount 2.59 sec 0.17 sec 2.25 sec
Empty 2.57 sec 0.11 sec 0.03 sec
Full 2.63 sec 0.16 sec 0.80 sec


  • IOZone results
UBIFS compared to YAFFS2 (file size = 4MB)
4M write rewrite read reread r.read r.write b.read r.rewrite s.read fwrite frewrite fread freread
yaffs2 1538 1527 296962 297696 297825 1521 296876 1526 295164 1536 1524 297436 299021
ubifs 19913 20989 298346 297631 299809 20968 298368 20181 299458 20471 20962 297371 297760
ubifs/yaffs2 12.95 13.75 1.00 1.00 1.01 13.79 1.01 13.22 1.01 13.33 13.75 1.00 1.00
UBIFS compared to YAFFS2 (file size = 16MB)
16M write rewrite read reread r.read r.write b.read r.rewrite s.read fwrite frewrite fread freread
yaffs2 1538 1523 297199 298525 298896 1528 298433 1525 298847 1538 1525 299661 300656
ubifs 20501 20656 298272 299109 299032 20521 298417 20710 298923 20448 20521 298259 299267
ubifs/yaffs2 13.33 13.56 1.00 1.00 1.00 13.43 1.00 13.58 1.00 13.30 13.46 1.00 1.00
UBIFS compared to YAFFS2 (file size = 32MB)
32M write rewrite read reread r.read r.write b.read r.rewrite s.read fwrite frewrite fread freread
yaffs2 1539 1525 296537 297204 297253 1527 297177 1527 296583 1531 1523 296537 297463
ubifs 20474 20611 296765 297490 297334 20659 296972 38416 297371 19068 20059 296618 296658
ubifs/yaffs2 13.30 13.52 1.00 1.00 1.00 13.53 1.00 25.16 1.00 12.45 13.17 1.00 1.00

Creating UBI Image

This is easiest to do, if you have access to the device and can run ubinfo and dmesg, otherwise you'll need to determine the volume size, Logical Erase Block size, etc by other means. UBI has some block overhead, which I found documentation inconsistent with my particular application, so your results may very. If your device is one UBI image for the entire NAND, this should be easier, and could probably be determined by just mounting a copy of the UBI image from the device if available.

To create the image from a rootfs you've built first you need to create the ubi.ini file, that describes your ubi image. Create a regular text file, ubi.ini, example contents, for more info run ubinize -h:

[ubi_rfs]
mode=ubi
image=ubifs.img
vol_id=0
vol_size=87349248
vol_type=dynamic
vol_name=ubi_rfs
vol_alignment=1
vol_flags=autoresize

Next you'll run the commands that actually build it. Here ubi.ini is the file you just created, ubifs.img is a temp file you can delete once you are done, and your_erootfs.ubi is the name of the rootfs image that will be created.

sudo /usr/sbin/mkfs.ubifs -m 2048 -e 129024 -c 677 -r /path/to/rootfs ubifs.img
sudo /usr/sbin/ubinize -o your_erootfs.ubi -p 131072 -m 2048 -s 512 -O 512 ubi.ini

To determine these and the ubi.ini file settings, use ubinfo -a and dmesg on the device if possible, which both give plenty of information about the values needed. The size and vol_name are listed under "Present volumes" when you run ubinfo -a on the device. The second half of that particular ubi device's description. While the NAND description's PEB, LEB etc are in dmesg.

mkfs.ubifs

  • -m - Minimum I/O unit size.
  • -e - Logical Erase Block (LEB) size.
  • -c - Max LEB count. (vol_size/LEB)
  • -x - Compression type: lzo (default), favor_lzo, zlib, none
  • -r - Path to root filesystem.
  • ubifs.img - Temporary image file.

ubinize

  • -o - Output file.
  • -p - Physical Erase Block (PEB) size.
  • -m - Minimum I/O unit size.
  • -s - Minimum I/O size for UBI headers, eg. sub-page size.
  • -O - VID header offset from start of PEB.
  • ubi.ini - UBI image configuration file.

Mounting UBI Image on PC using nandsim

First create a simulated NAND device (this one is 256MB, 2048 page size). <number>_id_byte= corresponds to the ID bytes sent back from the NAND.

$ sudo modprobe nandsim first_id_byte=0x20 second_id_byte=0xaa third_id_byte=0x00 fourth_id_byte=0x15

Check it was created.

$ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 10000000 00020000 "NAND simulator partition 0"

Next, attach it to a mtd device.

$ sudo modprobe ubi mtd=0

I had to detach it prior to formatting it.

$ sudo ubidetach /dev/ubi_ctrl -m 0

If that ubidetach step fails when you enter it, just proceed to the next step to format the mtd device.

$ sudo ubiformat /dev/mtd0 -f <image>.ubi
ubiformat: mtd0 (nand), size 268435456 bytes (256.0 MiB), 2048 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 2047 -- 100 % complete  
ubiformat: 2048 eraseblocks have valid erase counter, mean value is 1
ubiformat: flashing eraseblock 455 -- 100 % complete  
ubiformat: formatting eraseblock 2047 -- 100 % complete  

Then, attach it.

$ sudo ubiattach /dev/ubi_ctrl -m 0
UBI device number 0, total 2048 LEBs (264241152 bytes, 252.0 MiB), available 0 LEBs (0 bytes), LEB size 129024 bytes (126.0 KiB)

Make a target directory, and mount the device.

$ mkdir temp
$ sudo mount -t ubifs ubi0 temp