[Bug 178678] navigating sshfs-mounted file systems is extremely slow in dolphin and konqueror compared to command line

[Peter Penz]

http://bugs.kde.org/show_bug.cgi?id=178678


Peter Penz peter penz gmx at changed:

What |Removed |Added
----------------------------------------------------------------------------
AssignedTo|peter.penz-***@public.gmane.org |faure-***@public.gmane.org
Component|general |general
Product|dolphin |kio
Version|unspecified |4.1

[d***@public.gmane.org]

http://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #1 from doc evans gmail com 2008-12-25 04:40:48 ---
I truly hate to second-guess developers, but are you absolutely certain that
this is a kio problem?

It's not at all obvious to me why it would be.

(The answer "Yes" will satisfy me that you have thought about it and concluded
that indeed it is; I just want to be sure that you aren't confusing this with
something to do with the fish kio_slave.)

[Thorsten Hirsch]

https://bugs.kde.org/show_bug.cgi?id=178678


Thorsten Hirsch <t.hirsch-S0/***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |t.hirsch-S0/***@public.gmane.org




--- Comment #2 from Thorsten Hirsch <t hirsch web de> 2010-03-24 00:51:01 ---
I can confirm this bug. It's still present in KDE 4.4.1.
Whenever I navigate a directory up- or downwards dolphin says "Connecting" and
"Initiating protocol" in the status bar. So the ssh session is obviously not
being kept open, but initiated from start for every directory change. That's
what makes navigating so slow.
Please implement sessions for ssh (fish) in kio or dolphin and konqueror!

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678


tony <lorefice2-VGgt2q2+T+***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |lorefice2-VGgt2q2+T+***@public.gmane.org




--- Comment #3 from tony <lorefice2 libero it> 2010-04-09 13:00:50 ---
4.4.2 here.

While i can confirm the slowness on sshfs filesystem (as well as cifs/smbfs), i
don't think this is related to kio.
It's just that dolphin seems to catch a lot of info out of files/directories...
syncronously, blocking the gui.
It seems a little better if i don't display some columns (like permissions,
owner and so on).

So, if the underlaying filesystem is slow, dolphin will hangs here and there.
Konqueror from 3.5 is far snappier.

[Martin Fernau]

https://bugs.kde.org/show_bug.cgi?id=178678


Martin Fernau <martin-xDVbpdbWj+***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |martin-xDVbpdbWj+***@public.gmane.org

[d***@public.gmane.org]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #4 from <doc evans gmail com> 2010-04-09 17:23:10 ---
I don't see how it can be an underlying filesystem problem.

Responsiveness is great from the command line, so it seems to me that it has to
be a KDE issue (whether it's a kio issue or not I'm less sure, but I'm as
certain as I can be that it's a KDE problem).

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #5 from tony <lorefice2 libero it> 2010-05-04 12:44:44 ---
Bug confirmed in kde 4.3.3.
Filesystem bugs can be excluded because
- the bug is present even when browsing an smbfs/cifs mounted shares
- konqueror from kde3.4 is much responsive

Please, vote for this bug.

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678


Antonio Orefice <kokoko3k-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |kokoko3k-***@public.gmane.org




--- Comment #6 from Antonio Orefice <kokoko3k gmail com> 2010-05-23 13:09:10 ---
I've made some benchmarks comparing dolphin,konqueror, krusader ,pcmanfm and
thunar, they have to Show a remotely mounted sshfs directory with 1997 items
(all folders) and scroll; all on a 10Mbps half duplex link.

* Terminal (ls -la): 1.313sec

* Dolphin (kde4.3.3): about 10 seconds (but scrolling is almost impossible as
dolphin seems to be frozen, it is even hard to close it)
* Konqueror (kde4.3.3): about 13 seconds (same issues with
scrolling/freezing,closing)
* Krusader (2.0.0 for kde4) with mimetype magic disabled: about 6 seconds
(scrolling is still a pain, but faster than konqueror and dolphin for kde4)


* krusader (1.9.0 for kde3): about 2 seconds (scrolling is very fast)
* pcmanfm: about 1.5seconds (no issues in scrolling/closing)
* thunar: about 1.5seconds (no issues in scrolling/closing)
* rox: about 1.5seconds (no issues in scrolling/closing)

Hoping this will help to clarify that issues with speed are *not (only)*
related to the filesystem, so i think this should be addressed as a kio problem
with slow filesystems.

[Stefan Schramm]

https://bugs.kde.org/show_bug.cgi?id=178678


Stefan Schramm <mail-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEW
Ever Confirmed|0 |1




--- Comment #7 from Stefan Schramm <mail stefanschramm net> 2010-08-01 14:13:34 ---
*** This bug has been confirmed by popular vote. ***

[Stefan Schramm]

https://bugs.kde.org/show_bug.cgi?id=178678


Stefan Schramm <mail-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |mail-***@public.gmane.org




--- Comment #8 from Stefan Schramm <mail stefanschramm net> 2010-08-01 14:31:04 ---
When running tcpdump you can see that many packets (> 100) are transferred just
when resizing the window or hovering above a directory.

[Alvaro Aguilera]

https://bugs.kde.org/show_bug.cgi?id=178678


Alvaro Aguilera <alvaro.aguilera-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |alvaro.aguilera-***@public.gmane.org




--- Comment #9 from Alvaro Aguilera <alvaro aguilera gmail com> 2010-12-03 10:51:07 ---
Dolphin is also very slow when copying files via ssh.

I did a simple test and it took dolphin 4 minutes and 20 seconds to copy what
you can copy with scp in just 2 minutes and 30 seconds. Twice as much.

[Mads]

https://bugs.kde.org/show_bug.cgi?id=178678


Mads <mads-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |mads-***@public.gmane.org

[Andrew Gaydenko]

https://bugs.kde.org/show_bug.cgi?id=178678


Andrew Gaydenko <***@gaydenko.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gaydenko.com

[Andrew Gaydenko]

https://bugs.kde.org/show_bug.cgi?id=178678


Andrew Gaydenko <***@gaydenko.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC|***@gaydenko.com |

[Andrew Gaydenko]

https://bugs.kde.org/show_bug.cgi?id=178678


Andrew Gaydenko <***@gaydenko.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gaydenko.com

[pier andre]

https://bugs.kde.org/show_bug.cgi?id=178678


pier andre <pier_andreit-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |pier_andreit-***@public.gmane.org

[DCh]

https://bugs.kde.org/show_bug.cgi?id=178678


DCh <dchilton+kde-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |dchilton+kde-***@public.gmane.org




--- Comment #11 from DCh <dchilton+kde bestmail us> 2011-04-18 16:47:14 ---
(In reply to comment #4)

Post by d***@public.gmane.org
I don't see how it can be an underlying filesystem problem.
Responsiveness is great from the command line, so it seems to me that it has to
be a KDE issue (whether it's a kio issue or not I'm less sure, but I'm as
certain as I can be that it's a KDE problem).

i'm seeing the same behavior in KDE 4.6.2. one additional data point --
kdevelop4

at shell, navigating sshfs mount is fast/responsive
in Dolphin, slow/sluggish, as reported here

but,

navigating same tree in kdevelop4 is fast/responsive.

for reference, i'm running

kdevelop4-4.2.0-46.7.x86_64
dolphin-4.6.2-5.1.x86_64

[Nikita Melnichenko]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #12 from Nikita Melnichenko <open-development yandex ru> 2011-04-22 19:56:12 ---
Created an attachment (id=59213)
--> (http://bugs.kde.org/attachment.cgi?id=59213)
fs requests of krusader reading /

FUSE debug mode shows all FS requests that krusader made

[Nikita Melnichenko]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #13 from Nikita Melnichenko <open-development yandex ru> 2011-04-22 19:56:43 ---
Created an attachment (id=59214)
--> (http://bugs.kde.org/attachment.cgi?id=59214)
fs requests of mc reading /

FUSE debug mode shows all FS requests that mc made

[Nikita Melnichenko]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #14 from Nikita Melnichenko <open-development yandex ru> 2011-04-22 19:56:52 ---
Created an attachment (id=59215)
--> (http://bugs.kde.org/attachment.cgi?id=59215)
fs requests of ls reading /

FUSE debug mode shows all FS requests that 'ls -la' command made

[Nikita Melnichenko]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #15 from Nikita Melnichenko <open-development yandex ru> 2011-04-22 20:29:57 ---
I used 'bindfs -d' to grab all requests when using different methods to list
directory. Results for 20 directories in the '/' are:

Krusader: 258 requests.
MC: 69 requests.
ls -la: 49 requests (but note that it tries to read xattrs)
ls: 27 requests (seems to be the optimal number, because it only calls getattr
for each element that is a 'lstat' call for each file, and it's sufficient to
form a full-featured listing)

So, the Krusader does to many excessive requests. Even if KDE programs need all
that /dir/.directory requests, it would be optimal to have only about 50
requests...

[s***@public.gmane.org]

https://bugs.kde.org/show_bug.cgi?id=178678


sushyad-***@public.gmane.org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |sushyad-***@public.gmane.org




--- Comment #16 from <sushyad gmail com> 2011-04-29 20:42:19 ---
I can confirm this on KDE 4.6.2, Dolphin 1.6.1

The remote directory is mounted using cifs with these options:
directio,rsize=130048

Command line copy from remote CIFS server is ~30 MiB/s
Dolphin copy from remote CIFS server is ~10MiB/s
Konqueror copy from remote CIFS server is ~10MiB/s

[DCh]

https://bugs.kde.org/show_bug.cgi?id=178678


DCh <dchilton+kde-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC|dchilton+kde-***@public.gmane.org |

[c***@public.gmane.org]

https://bugs.kde.org/show_bug.cgi?id=178678


carbonfreeze-***@public.gmane.org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |carbonfreeze-***@public.gmane.org

[Thomas Langkamp]

https://bugs.kde.org/show_bug.cgi?id=178678


Thomas Langkamp <langkamp-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |langkamp-***@public.gmane.org




--- Comment #17 from Thomas Langkamp <langkamp tomblog de> 2011-10-14 22:31:43 ---
I want to confirm this bug, using kde 4.7.2 with opensuse tumbleweed 64bit and
nfs4 transfers under dolphin. Dolphin freezes for up to 30 seconds then
responds for 5 seconds, then freezes for 30 seconds and so on...

[pier andre]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #18 from pier andre <pier_andreit yahoo it> 2011-10-15 10:54:34 ---
Me too want to confirm this bug, using kde 4.7.2 with opensuse tumbleweed
64bit. if I transfer large files (10 movies 0.7GB each) using sftp://...
protocol under dolphin, transfere starts, but if, after the transfere is
started, I try to change directory dolphin freezes for minutes, in practice
during transfering dolphin is unusable in that sftp://, if I use konqueror it
doesn't happen.
I would like to point light on this bug that is very ugly.

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #19 from tony <lorefice2 libero it> 2011-10-15 10:58:42 ---
Please, do not mix bugs.
We're talking about mounted and (relatively) slow network filesystems.
kio_sftp doesn't mount anything, nor tries to read a lot of attributes from the
files.

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678


David Faure <faure-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |afiestas-***@public.gmane.org,
| |faure-***@public.gmane.org




--- Comment #20 from David Faure <faure kde org> 2011-10-28 17:11:32 ---
Obviously KIO thinks the file system is "local" while in fact it's remote. So
it does operations that end up being too slow.
I initially thought about previews and mimetype determination, but the first
one can be turned off in the menus, and the second one is delayed anyway.

Slow copying is another issue, possibly fixed by the much more recent commit
2cd2d1a4cfa1 on Sep 21, 2011, bug 257907 and bug 258497. Let's only talk about
listing and reloading here, the bug title says "navigating".

[My testcase for future reference: sshfs www.davidfaure.fr: /mnt/df ; cd
/mnt/df/txt.git/objects ]

Ah! With some breaking-in-gdb (poor-man's profiling), I found it. What's really
slow is subdirectories, in detailed view mode, because it lists every subdir in
order to display the number of items. And this it does NOT do in a delayed
manner:

#0 0x00007f76af211b0a in __getdents64 () from /lib64/libc.so.6
#1 0x00007f76af2114d1 in readdir64 () from /lib64/libc.so.6
#2 0x00007f76b46d0dba in KDirModel::data (this=0xccfce0, index=...,
role=743246400) at /d/kde/src/4/kdelibs/kio/kio/kdirmodel.cpp:739
#3 0x00007f76a3809f8f in DolphinModel::data (this=0xccfce0, index=...,
role=743246400) at
/d/kde/src/4/kde-baseapps/dolphin/src/views/dolphinmodel.cpp:119
#4 0x00007f76b179436d in QSortFilterProxyModel::data (this=0xcdcd50,
index=..., role=743246400) at itemviews/qsortfilterproxymodel.cpp:1716
#5 0x00007f76b46f51cd in QModelIndex::data (this=0x7fff1e66f0a0,
arole=743246400) at
/d/qt/4/qt-for-trunk/include/QtCore/../../src/corelib/kernel/qabstractitemmodel.h:398
#6 0x00007f76b46ed564 in KFileItemDelegate::Private::itemSize (this=0xcaa6c0,
index=..., item=...) at /d/kde/src/4/kdelibs/kio/kio/kfileitemdelegate.cpp:226
#7 0x00007f76b46f11e3 in KFileItemDelegate::Private::display (this=0xcaa6c0,
index=...) at /d/kde/src/4/kdelibs/kio/kio/kfileitemdelegate.cpp:987
#8 0x00007f76b46f0640 in KFileItemDelegate::Private::initStyleOption
(this=0xcaa6c0, option=0x7fff1e66e800, index=...) at
/d/kde/src/4/kdelibs/kio/kio/kfileitemdelegate.cpp:857
#9 0x00007f76b46f2027 in KFileItemDelegate::paint (this=0xcaa510,
painter=0x7fff1e66f570, option=..., index=...) at
/d/kde/src/4/kdelibs/kio/kio/kfileitemdelegate.cpp:1252

Fixed (skipped for nfs/smb mounts), see commit in next comment.

Then there is the reading of icon name (and comment) in "foo/.directory"
(KFileItem::iconName calling KFolderMimeTypePrivate::iconName).
Even though it's cached (so it happens only once per dir), it still makes
things quite slow.
=> Fixed too (skipped for nfs/smb mounts).

The last issue seems to be when navigating away from a directory,
KDirListerCache::forgetDirs calls manually_mounted for -each item-, which calls
realFilePath, which stats... There's already a TODO for porting that code to
Solid, assigned to afiestas ;-)

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #21 from David Faure <faure kde org> 2011-10-28 17:17:09 ---
Git commit 02b7eb5d92daae4373e7d38d2d952a688bd42079 by David Faure.
Committed on 28/10/2011 at 19:12.
Pushed by dfaure into branch 'KDE/4.7'.

Detect network mounts and skip slow operations while listing them.

On NFS/SMB/SSHFS, KIO no longers counts items in directories (e.g.
for dolphin in detailed view mode), nor does it look at subdir/.directory
files for custom icons or comments.

This commit will improve speed greatly in kdelibs-4.7.4 and later.
There's just one issue remaining, when navigating away from a directory
with many items, but that's a TODO for afiestas -- if he agrees :)

CCBUG: 178678

M +16 -1 kdecore/io/kfilesystemtype_p.cpp
M +5 -3 kdecore/io/kfilesystemtype_p.h
M +2 -1 kio/kio/kdirmodel.cpp
M +43 -9 kio/kio/kfileitem.cpp
M +7 -0 kio/kio/kfileitem.h
A +55 -0 kio/kio/kfileitemaction_twomethods-reverted.diff

http://commits.kde.org/kdelibs/02b7eb5d92daae4373e7d38d2d952a688bd42079

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #22 from tony <lorefice2 libero it> 2011-10-28 17:31:38 ---
Thank you for looking and fixing this;
I looked a bit to the code, wouldn't be better to leave some room for other
filesystems too?
I mean that as far as i understood, non "normal" filesystems seems to be
hardcoded in kfilesystemtype_p; what if i would use an exotic one?
There's still curlftpfs, httpfs and god only knows how many fuse
implementations :)
I'm not a c coder and i don't know how much trouble it could cause, but what
about a .rc file filled with slow filesystems?

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #23 from David Faure <faure kde org> 2011-10-29 07:38:49 ---
Sure, feel free to give me a list of "network mounts" filesystems :-)

A config file can't do, because there are two implementation of
KFileSystemType: one which looks at the string returned by mount (the code
where I added fuse.sshfs), and one (faster, and used on linux) which looks at
statfs.f_type "magic numbers", see the code where I added SSHFS_SUPER_MAGIC. So
to add detection of new filesystem types, I need "super-magic" numbers as well.

I still want the fallback for unknown filesystems to be "local and normal",
since that's the most feature-full case, and people seem to come up with new
local filesystems all the time.

curlftpfs and httpfs are based on fuse like sshfs, right? I guess I could
provide a small test prog that outputs the statfs.f_type value for a given
directory, unless someone can find a documentation about these values
somewhere...

[aditsu]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #24 from aditsu <aditsu yahoo com> 2011-10-29 10:37:23 ---
If possible, I suggest automatically detecting all filesystems based on fuse.
Also, it would be nice if the user could manually switch between "local" and
"network" browsing modes for a location.

[Heinz Wiesinger]

https://bugs.kde.org/show_bug.cgi?id=178678


Heinz Wiesinger <HMWiesinger-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |HMWiesinger-***@public.gmane.org




--- Comment #25 from Heinz Wiesinger <HMWiesinger gmx at> 2011-10-29 10:53:38 ---
(In reply to comment #24)

Post by aditsu
If possible, I suggest automatically detecting all filesystems based on fuse.

You can't generalize like that. There are local filesystems based on fuse as
well (like ntfs-3g). Treating them as remote filesystems would just be plain
wrong.

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #26 from tony <lorefice2 libero it> 2011-10-29 11:01:13 ---

Post by David Faure
Detect network mounts and skip slow operations while listing them.

Is 'skip' intended as 'never do it' or just 'delay it'?
Because in the second case the logic could be used for network and local
filesystems too (?)

[tony]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #27 from tony <lorefice2 libero it> 2011-10-29 11:09:38 ---
(In reply to comment #23)

Post by David Faure
A config file can't do, because there are two implementation of
KFileSystemType: one which looks at the string returned by mount (the code
where I added fuse.sshfs), and one (faster, and used on linux) which looks at
statfs.f_type "magic numbers"

Couldn't that file be in the format:
[network_filesystems_strings]
.
.
.
[network_filesystems_super_magic]
.
.
.
(EOF)
...so that the first that matches automatically identifies the fs type?

I intend the use of that file just as a fallback where the hardcoded values are
unable to identify the filesystem, so the performance impact should not be high
and the user will not need to wait for another bug report to be closed and
another kde-libs to be released to add support to a newly created fs.

[Alex Fiestas]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #28 from Alex Fiestas <afiestas kde org> 2011-10-29 17:54:39 ---

Post by David Faure
This commit will improve speed greatly in kdelibs-4.7.4 and later.
There's just one issue remaining, when navigating away from a directory
with many items, but that's a TODO for afiestas -- if he agrees :)

I do agree but, what do I have to do ?

Your words my commands.

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #29 from David Faure <faure kde org> 2011-10-31 10:51:40 ---
Tony: and how would you figure out the "super magic" for these file systems,
when I couldn't even find it in the header where these are usually defined, so
I had to reverse-engineer it?
Anyway, KDE is already very configurable, but I don't think this needs a config
file, i.e. delegating the problem to users. This should rather be done right in
the code, so that it only needs to be solved once, not by every user. Whatever
you would have put in that config file, tell me, and I'll put it in the code
:-)

Alex: see `grep afiestas kdirlister.cpp`

[Michael Berlin]

https://bugs.kde.org/show_bug.cgi?id=178678


Michael Berlin <michael.berlin.xtreemfs-gM/Ye1E23mwN+***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@goo
| |glemail.com




--- Comment #30 from Michael Berlin <michael berlin xtreemfs googlemail com> 2011-11-08 10:07:53 ---
Hi,

I'm the developer of the fuse client of the distributed file system XtreemFS
(see www.xtreemfs.org for more information). Can you please also include our
file system and recognize it as "network file system"?

Our client reports a file system type of the form "xtreemfs@<server url>", so
checking for "xtreemfs@" in the beginning would be sufficient:

$ df -h /mnt
Filesystem Size Used Avail Use% Mounted on
xtreemfs-kySbmCM36Nw4v9Oa8djDPti2O/***@public.gmane.org/demo
20G 7.0G 13G 36% /mnt

Btw: I cannot tell you another super magic number than the one sshfs does
report. See:

$ strace stat -f /mnt 2>&1|grep statfs|grep mnt
statfs("/mnt", {f_type=0x65735546, f_bsize=131072, f_blocks=161229,
f_bfree=104645, f_bavail=104645, f_files=2048, f_ffree=2048, f_fsid={0, 0},
f_namelen=1024, f_frsize=131072}) = 0

From my file system in userspace I don't have the possibility to set the struct

member f_type. Instead I've found this super magic number in the Fuse kernel
code:
http://lxr.free-electrons.com/source/fs/fuse/inode.c#L50

So I guess you'll have to rely on the file system type to differ between fuse
implementations.

Best regards,
Michael

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #31 from David Faure <faure kde org> 2011-11-08 10:20:58 ---
Git commit 8b57d2b80329c9d005145354bdd5db8de3d6ede6 by David Faure.
Committed on 08/11/2011 at 11:20.
Pushed by dfaure into branch 'KDE/4.7'.

Sigh, with FUSE nothing is ever simple.

Thanks to Michael Berlin for the information about that. Add xtreemfs.
CCBUG: 178678

M +7 -5 kdecore/io/kfilesystemtype_p.cpp

http://commits.kde.org/kdelibs/8b57d2b80329c9d005145354bdd5db8de3d6ede6

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #32 from Antonio Orefice <kokoko3k gmail com> 2011-11-08 11:04:03 ---
Looking at the code, it seems that curlftpfs is still missing, and from what i
understood fuse has just one magic number.
So, heres how a filesystem using curlftpfs appears in my mounts:

Gozer ~ # mount|grep curlftpfs
curlftpfs#ftp://s1/ on /mnt/ftp type fuse (rw,nosuid,nodev,noatime)
curlftpfs#ftp://s2:212/T/ on /mnt/nasftp type fuse (rw,nosuid,nodev,noatime)

thanks!

[Andrew Gaydenko]

https://bugs.kde.org/show_bug.cgi?id=178678


Andrew Gaydenko <***@gaydenko.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC|***@gaydenko.com |

[Chassagneux]

https://bugs.kde.org/show_bug.cgi?id=178678


Chassagneux <maxime.chassagneux-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |maxime.chassagneux-***@public.gmane.org
| |m




--- Comment #38 from Chassagneux <maxime chassagneux gmail com> 2012-01-13 13:21:11 ---
I think that the Markus Grabner proposal is really good !
I've opened this bug 290680, that is related to this thread, and your
workaround could fix it too !

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #39 from Antonio Orefice <kokoko3k gmail com> 2012-01-13 13:27:43 ---
I agree, having a full async filemanager (not only for thumbnails) would be
perfect.

[Alvaro Aguilera]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #40 from Alvaro Aguilera <alvaro aguilera gmail com> 2012-01-13 13:31:12 ---
How about just stopping non-critical processes after timeout instead of moving
them to the background? I don't like to push the dust under the carpet.

[Chassagneux]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #41 from Chassagneux <maxime chassagneux gmail com> 2012-01-13 13:33:55 ---
As these processes are async why not...

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #42 from Antonio Orefice <kokoko3k gmail com> 2012-01-13 13:36:33 ---
@#39:
It would not be like push the dust under the carpet, but more like to trow it
away as you walk.
What i imagine is just like thumbnails view, where what is really updated is
just what is needed (aka what you see)

[Alvaro Aguilera]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #43 from Alvaro Aguilera <alvaro aguilera gmail com> 2012-01-13 13:39:10 ---
Because it would be easier to implement and less error prone. This bug was
opened in 2008 and is still unresolved. Going fully async. would probably
create more problems than it solves.

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #44 from Antonio Orefice <kokoko3k gmail com> 2012-01-13 13:43:45 ---
...probably you're right,
but dropping features because they're not easy to implement is (imho) a big
lose.

[Antonio Orefice]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #45 from Antonio Orefice <kokoko3k gmail com> 2012-01-13 13:46:02 ---
Anyway a big step forward in identify some network filesystem has been done,
dolphin now is much snappier here with 4.7.4, so there would be no (so much)
hurry anymore.

[Chassagneux]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #46 from Chassagneux <maxime chassagneux gmail com> 2012-01-13 13:49:01 ---
But since the 4.7.4, a regression with the personal icon display feature was
disappeared !

[Elias Probst]

https://bugs.kde.org/show_bug.cgi?id=178678


Elias Probst <mail-I6wIA1cK5b6nCEr/***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Blocks| |290680

[k***@public.gmane.org]

https://bugs.kde.org/show_bug.cgi?id=178678


knallio-***@public.gmane.org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |knallio-***@public.gmane.org




--- Comment #47 from <knallio gmx de> 2012-01-24 12:09:00 ---
We have NFS mounted \home directories in our office with Gbit network. We never
had any performance issues, but now our users are complaining that the number
of elements of a folder are not displayed anymore. So the possibility to
enable/disable the directory listing for NFS would be really great.

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678


David Faure <faure-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |peter.penz19-***@public.gmane.org




--- Comment #49 from David Faure <faure kde org> 2012-02-06 14:19:22 ---
Re comments 33, 34, and 36 : this looks like a bug in folderview. If it's still
not fixed, please report it there, I don't maintain that code.

About the idea of making this delayed: yes it's probably the best solution. We
already do delayed mimetype detection (for cases where the extension is unknown
or ambiguous), we could also do delayed ".directory-file loading". Peter, what
do you think? I'm not sure how much dolphin is still using kdelibs for the
delayed mimetype determination stuff...

[Peter Penz]

https://bugs.kde.org/show_bug.cgi?id=178678





--- Comment #50 from Peter Penz <peter penz19 gmail com> 2012-02-06 14:50:08 ---

Peter, what do you think? I'm not sure how much dolphin is
still using kdelibs for the delayed mimetype determination stuff...

@David: Dolphin only uses KFileItem and KDirLister from kdelibs. So in case if
we do a delayed mimetype detection it would absolutely fine from a Dolphin
point of view that the .directory file is read asynchronously from kdelibs and
the icon is updated in Dolphin later. If you have time to implement this,
please just forward me the patch so that I can check it in the scope of Dolphin
(or keep me on CC when pushing the patch). Thanks!

[Johannes Huber]

https://bugs.kde.org/show_bug.cgi?id=178678


Johannes Huber <johu-aBrp7R+bbdUdnm+***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |johu-aBrp7R+bbdUdnm+***@public.gmane.org

[Stef Bon]

https://bugs.kde.org/show_bug.cgi?id=178678


Stef Bon <stefbon-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |stefbon-***@public.gmane.org




--- Comment #51 from Stef Bon <stefbon gmail com> 2012-03-01 17:53:40 ---
HI,

I'm working with a FUSE fs and experienced the same. The filesystem provides by
design a .directory which fits at the target, like a smb share, with a smb
share icon and like a hard disk, with a harddrive icon, and a usb stick, also
with a removable pendrive icon. This is not slow at all, so the very strickt
choice to not try to process the .directory is not the best option in my
opinion.

In my opinion a heuristic manner, like described in comment 37 is better. And
maybe some support of other services as well. I develop a successor of gamin,
the filesystem change notifier, and think of enabling some caching abilities.

Caching the .directory file is one of the possibilities....

Stef

[Stef Bon]

https://bugs.kde.org/show_bug.cgi?id=178678

--- Comment #52 from Stef Bon <stefbon-***@public.gmane.org> ---
Hi,

I've been looking for a problem of this issue. I aggree that a backthround
thread or other process doing the (slow) reading of mimetype and .directory
files might work and bring the reading a lot faster.

But the problem here is what causing this. First the determination of the
filetype (mimetype for regular files). It can be slow, and doing that over and
over again s silly. Why not use the field st_rdev part of the stat struct. It's
not used for regular files, and large enough to hold an unique id to a
(standard) database of filetypes.

This st_rdev attribute is only used special files as far as I know, and is zero
for regular files and directories. On my machine the type is or unsigned long
int, or better unsigned long long int.

In any case it's big enough.

It's safe to use an id, A mp3 file will not turn in a ogg file suddenly, a pdf
will never become a doc or something else.
Apps creating these files can set this value, and apps using the file can set
it when it's not been set before, or correct it when not set to the right
value.

When using this value, there is no other process required, it comes to an app
like dolphin just like the mode (filetype, owner, group, permissions, times)
does.

Futher when it comes to directories, the reading of the .directory file is a
time consuming task. In my FUSE fs, I'm using the mimetype for directories
like:

mimetype="inode/directory;role=remote.net.smb.network"

for the directory representing the SMB network for example. More examples:

local.map.system.programs
local.dev.cdrom.audio
local.map.documents
local.map.audio

etc.

Because the mimetype gives you a default icon (just like this mimetype does for
regular files) it's not required to use a .directory file for setting and using
the .directory file for an icon.
And it's possible to store this mimetype for directories also in the st_rdev
field.

I'm not saying that .directory files are useless, but using the mimetype for
directories can speed up things. And - as I see it - directories can have a
special meaning within the context of the computer or the user, and till now
there is no way to set this. Using the role part of the mimetype makes that
possible.

Stef

[Jekyll Wu]

https://bugs.kde.org/show_bug.cgi?id=178678

Jekyll Wu <adaptee-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Blocks| |291138

[Jekyll Wu]

https://bugs.kde.org/show_bug.cgi?id=178678

Jekyll Wu <adaptee-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Blocks| |290666

[Bjoern Olausson]

https://bugs.kde.org/show_bug.cgi?id=178678

Bjoern Olausson <spamsuxx-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |spamsuxx-***@public.gmane.org

--- Comment #55 from Bjoern Olausson <spamsuxx-***@public.gmane.org> ---
This issue drives me crazy. Browsing a NFS/CIFS share which is mounted via my
DSL Internet connection is no fun - I have to do it multiple times a day...

A simple option which prevents Dolphin from reading the mime types and
especially recursively counting the numbers of items (which is totally
uninteresting by the way) in the displayed folders would do the trick for me -
removing the tab does in my test not stop Dolphin from counting...

Cheers,
Bjoern

[Alvaro Aguilera]

https://bugs.kde.org/show_bug.cgi?id=178678

--- Comment #56 from Alvaro Aguilera <alvaro.aguilera-***@public.gmane.org> ---
I think Dolphin's developers should use a slow, high latency connection to a
remote folder-tree with thousands of files as their _primary_ test environment.
If a feature works there well, then it will also shine with the local SSD
drive. And if it doesn't, then it's probably a bad idea to add that feature.

[Evstifeev Roman]

https://bugs.kde.org/show_bug.cgi?id=178678

Evstifeev Roman <someuniquename-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |someuniquename-***@public.gmane.org

--- Comment #57 from Evstifeev Roman <someuniquename-***@public.gmane.org> ---
Just bumped into this problem with a smb directory, mounted with mount.cifs.
There are 10k avi video files there, and opening it in dolphin shows empty
directory for more than 3 minutes, while loading. (this is over wi-fi
connection)
Opening same directory with KIO (smb:// link) loads and show directory contens
iteratively, but still takes over 3 minutes to load completely

[David Faure]

https://bugs.kde.org/show_bug.cgi?id=178678

David Faure <faure-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|CONFIRMED |RESOLVED
Resolution|--- |FIXED

Git commit 6369b556ae9beef6888699d23b91326bac950ba4 by David Faure.
Committed on 27/03/2013 at 14:29.
Pushed by dfaure into branch 'KDE/4.10'.
Implement delayed loading of slow icons (from .desktop or .directory files)
Thanks to Pierre Sauter for making me fix this, and for testing that it
works over sshfs.
FIXED-IN: 4.10.2
M +19 -3 kio/kio/kfileitem.cpp
M +6 -0 kio/kio/kfileitem.h
M +2 -0 kio/tests/kfileitemtest.cpp
http://commits.kde.org/kdelibs/6369b556ae9beef6888699d23b91326bac950ba4

[Jekyll Wu]

https://bugs.kde.org/show_bug.cgi?id=178678

Jekyll Wu <adaptee-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Version Fixed In| |4.10.2
Latest Commit| |http://commits.kde.org/kdel
| |ibs/6369b556ae9beef6888699d
| |23b91326bac950ba4

[Julian Kalinowski]

https://bugs.kde.org/show_bug.cgi?id=178678

Julian Kalinowski <julakali-***@public.gmane.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |julakali-***@public.gmane.org

--- Comment #60 from Julian Kalinowski <julakali-***@public.gmane.org> ---
I was just experiencing a similar problem when browsing a CIFS-mounted share
with a slow implementation of the dfree_command (using greyhole storage
pooling, but the only thing that matters here is that counting free disk space
is relatively slow).

When i use dolphin to access a folder on the CIFS-mounted share, the disk free
command gets called once for every subfolder in detail view or when you
mouse-over so the information side bar is refreshed, even though it is the same
filesystem and the information is never displayed to the user.
I think this behaviour is a bug.

This doesn't matter with a fast df command, as samba provides by default, but
with a 0.1s execution time command (greyhole), it really slows navigating down.

[e***@public.gmane.org]

https://bugs.kde.org/show_bug.cgi?id=178678

empire-Bv7XYG+6kbHQT0dZR+***@public.gmane.org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |empire-Bv7XYG+6kbHQT0dZR+***@public.gmane.org

--- Comment #65 from empire-Bv7XYG+6kbHQT0dZR+***@public.gmane.org ---
Yes please, fix this.

NO metadata loading at all on networked drives. It's pretty straightforward.
The delays over a slow WAN are huge and not worth it.

[Nate Graham]

https://bugs.kde.org/show_bug.cgi?id=178678

--- Comment #66 from Nate Graham <***@kde.org> ---
*** Bug 215953 has been marked as a duplicate of this bug. ***

[Nate Graham]

https://bugs.kde.org/show_bug.cgi?id=178678

Nate Graham <***@kde.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Component|general |general
Assignee|***@kde.org |dolphin-bugs-***@kde.org
CC| |***@kde.org,
| |***@kde.org
Summary|navigating sshfs-mounted |Navigating mounted network
|file systems is extremely |locations is extremely slow
|slow in dolphin and |in Dolphin compared to
|konqueror compared to |command line
|command line |
Product|kio |dolphin
Keywords| |usability
Version|4.1 |18.04.0

[Thomas Guymer]

https://bugs.kde.org/show_bug.cgi?id=178678

Thomas Guymer <***@thomasguymer.co.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@thomasguymer.co.
| |uk

[Alex Fiestas]

https://bugs.kde.org/show_bug.cgi?id=178678

Alex Fiestas <***@kde.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC|***@kde.org |